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Abstract 


Medical students require higher-order thinking skills to diagnose patient cases. 

In order to develop expert clinical reasoning skills, students require instructional 
support; however, activities explicitly facilitating the development of such skills are 
often overlooked in problem-based learning tutorials. Collaboratively constructed 
diagrams could provide a framework where students can identify, analyse, interpret 
and evaluate information from a patient case to develop a diagnosis. A paper proto- 
type for a diagram construction toolkit was developed and tested with 13 students 
and 4 instructors. Data was collected and analysed to determine if medical students, 
with no formal visual training, can construct meaningful diagrams of a patient case 
when they are provided the right tools, and if the constructions can demonstrate 
tasks associated with higher-order thinking. Suggestions for modifications to create 
a computer-based version are recommended and future research opportunities are 


presented. 
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Introduction 


This document is written in support of a Master of Design thesis in Visual Com- 
munication Design. It reports on the process of developing an educational resource 
that uses collaboratively constructed diagrams as a framework for groups of medical 
students working with a complex problem such as the diagnosis of a patient’s illness. 

Wakefield (1996) stated that solving complex problems is a challenge that re- 
quires a student to engage in higher-order thinking processes to “strategically acquire 
knowledge to find a solution to a problem” (p. 408). Complex problems require 
more than just the memorization and recall of facts—students must be skilled at the 
tasks of identifying pertinent information, analysing and interpreting information 
within the scope of the problem, evaluating possible solutions and implementing the 
best option. 

Diagrams are a means of visually representing complex information structured 
in a way that enables the viewer to efficiently process information through identifica- 
tion, analysis, interpretation, and evaluation so a solution can be found (Figure 1). 
The operations just described characterize higher-order thinking and it appears that 
the collaborative construction of diagrams could be beneficial for the development 
of skills associated with higher-order thinking. A question, however, emerges: How 
can students, without formal visual training, create diagrams that are meaningful for 
the purpose of solving a complex problem? 

This thesis explores the issues involved in the development of a set of tools to 
enable users to work together to efficiently create meaningful diagrams that can be 


used to solve complex problems. 


FIGURE 1: Visual representations of complex information 


QUANTITY 


Diagrams can visually represent information in a way that makes the tasks of problem solving and 
decision more efficient. The map on the left shows data numerically and it is difficult to interpret 
due to the complexity. The one on the right uses proportionate sizes of dots to express quantity. 
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Frascara (1997) stated that visual communication design is an “activity directed 
at affecting the knowledge, the attitudes, and the behaviour of people” to improve 
their lives (p. 3). While designers must possess mastery of visual language to commu- 
nicate, they must also be skilled problem-identifiers and problem-solvers. Through 
the combination of these skills, designers are able to create effective communications 
that are not only apparent and attractive, but also understandable and convincing to 
the audience while considering their preferences and abilities (Frascara, 1997). The 
paradigm of creating effective communications with consideration of the needs of the 
user is called user-centred design. In user-centred design, it is essential that the com- 
munication is developed in collaboration with experts from relevant domains and 


with the participation of the audience toward whom the communication is directed. 


DESIGN PROCESS 


The simple existence of a visual design product, developed in partnership with the 
user, is not enough to indicate success. The designer follows a process that involves 
the planning, structuring, production and evaluation of communications (Frascara, 
2004). By recording the process, the design action becomes visible, the mystery that 
often accompanies a creative act is exposed, designers become accountable for their 
decisions and the propositions of designers become legitimized (Swann, 2002). 
Although there is no all-purpose design process that can apply to every activity, a 
general model has been established through years of practice and refinement. Swann 
(2002) described the model of the design process as: problem-analysis-synthesis-exe- 
cution-production-evaluation (p. 53). This model is emblematic of approaches used 
by design practitioners and researchers, and similar models can be found within the 
field of instructional design (Reeves & Hedberg, 2003; Kemp et al., 1998). Although 
the process appears linear, it is instead an iterative cycle where the results of analysis, 
reflection and evaluation stages influence the previous stages (Figure 2). Swann’s ac- 
tion and design research model is used as the basis for describing the activities that 


occurred during the course of this study. 


PROCESS SUMMARY 


BACKGROUND RESEARCH 


* Experience 
Six years of experience as an educational electronic media designer in the Faculty of 
Medicine and Dentistry at the University of Alberta made the researcher aware of 


certain instructional needs within the Faculty. Knowledge of the lack of tutors to 


Ficure 2: Action research and design process 
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A general design process has been established through years of practice and refinement. Although 
the process appears linear, it is instead an iterative cycle where the results of analysis, reflection 
and evaluation stages influence the previous stages 


facilitate small group, problem-based learning stimulated interest in finding a 


solution for the problem that involved visual communication design. 


- Consultations 

A series of consultations was conducted with faculty members in the Faculty of 
Medicine and Dentistry, instructors and students to help determine the needs of the 
institution, The Associate Dean of Undergraduate Medical Education, the Associate 
Director of the Division of Studies in Medical Education, two teaching block coor- 
dinators, the health sciences librarian, an expert in evidence-based medicine and 
medical informatics, and two second-year medical students were consulted. These 


consultations helped to frame the literature review. 


» Literature review 
The literature review provided background material to help the researcher under- 
stand the larger problem and served to contextualize comments made by individuals 


who were interviewed. 


PROBLEM 

In medical education, students are required to develop advanced higher-order think- 
ing skills to be able to deal with a complex problem like the diagnosis of a disease. 
The evaluation of a patient’s illness, identification of symptoms through reasoning 
and research, and generation of a correct diagnosis of the medical problem, in an 
efficient and humane manner, are a demanding set of skills for a student to develop. 


Problem-based learning is a teaching method utilized at many medical schools where 


Go 


autonomous groups of students can construct their personal understanding of a 
problem by sharing the knowledge and perspectives of others to solve an 
authentic task. 

While problem-based learning is intended to help students develop better 
higher-order thinking skills, research has shown it does not always enable students 
to become better at clinical reasoning and problem solving. In fact, studies have in- 
dicated that students from schools that incorporate problem-based learning in the 
curriculum are more likely to employ backwards reasoning—a characteristic of nov- 
ices—which starts with a possible solution and finds evidence to support the conclu- 
sion. In contrast to novices, experts tend to look at the whole picture before develop- 
ing a conclusion about an illness. 

In a clinical context an expert will reason through a medical case in the 


following sequence: 


1. The patient is interviewed to collect a health history relevant to their complaint; 
2. A list of diagnostic hypotheses, called a differential diagnosis, is developed; 


3. A physical exam is performed to collect more information to refine the 


differential diagnosis list; and 


4. Laboratory and diagnostic tests will be ordered to confirm the hypothesis. 


Presently, problem-based learning tutorials rely on a verbal exchange, which is 
transitory in nature. Methods of representing complex information visually, in the 
form of diagrams, have long been established as devices that aid the reader in the 
interpretation of data which can inform a decision making process. Due to the struc- 
ture of language, information obtained aurally or through text descriptions pro- 
motes a linear and sequential way of thinking. Diagrams, on the other hand, display 
text in a way that allows a reader to engage in a browse and search strategy that leads 
to viewing the entire set of data at once and selectively examine particular elements. 
Frascara (2001) claimed that diagrams permit “hierarchies, inclusions, simultaneity, 
distinctions of levels, multiplicity of kinds and complexity of connections” that can ex- 
pand our capacity to understand serious problems of a physical or social nature (p. 3). 

As devices that display complex sets of information that lead to better problem 
solving, diagrams serve a valuable function in education. Contemporary educational 
paradigms put an emphasis on the development of higher-order thinking in stu- 
dents. Higher-order thinking requires students to be adept at analysing, synthesiz- 
ing and evaluating new information and incorporating it with prior knowledge so 
they become better at solving problems, thinking critically and making decisions. 


When information is displayed in a diagram, students can perform these cognitive 


operations with much greater efficiency. Diagrams can serve as an “artificial graphic 
memory” where students are no longer required to store and recall information in 
order to process it (Bertin, 1983). 

The collaborative construction of diagrams of a patient case could display pa- 
tient information, and provide a permanent record of the activity thereby helping 
in the analysis, interpretation, and evaluation. With a diagrammatic representation 
of student work, tutors would be better able to assess the students’ ability to think 
critically, solve the problem and make decisions. 

By providing a focus for group activity, diagrams may also reduce the tenden- 
cies of some tutors who lecture or students who dominate the discussion and direct 
it onto topics tangential to the case at hand—something about which students fre- 
quently complain. By serving as a permanent record of group activities, diagrams of 
a patient case constructed by students can provide a greater degree of accountability 
and autonomy for members of small groups, thereby reducing the amount of face to 
face time required to conduct problem-based learning tutorials. 

Evidence found in the literature review supported the notion that a computer- 
based software application that uses diagrams to help students identify the compo- 
nents of a problem and work toward a solution could help develop higher- 
order thinking. 

Workstations linked together over computer networks would enhance the power 
of the proposed web-based learning application while serving the same purpose. 
Communication technology, such as instant messaging or discussion groups, could 
be provided to supplement the diagram construction activities of the web applica- 
tion to provide a more flexible structure for the group’s activities. Group members 
could be freed of the constraints of distance and time and they could decide to work 
as a group in front of a single computer, together, from the comfort of their own 
homes or whenever it is most convenient for them. The computer would be able to 
store the data permanently so that it could be accessed and edited by members of a 
group at any time. 

Incorporating a web application, such as the one proposed, into a learning situ- 
ation would then allow the possibility of returning problem-based learning tutorials 
to their original purpose: one where autonomous groups of students work together 
to solve a patient case. The tutor provides feedback to help students become better at 


solving problems, thinking critically and making decisions. 


ANALYSIS AND SYNTHESIS 
Separate elements of information collected from the background research and 


through consultations with stakeholders was combined into a coherent problem and 


it was determined that an instructional tool could address the needs of students and 


instructors if it was based on three premises: 


1. Problem-based learning tutorials enable medical students to develop higher- 
order thinking skills by providing an opportunity to work on an authentic task 


as part of an autonomous small group. 


2. Visual representations can organize and display complex information in a way 
that helps individuals efficiently identify, analyse, interpret and evaluate to 


diagnose a problem. 


3. Computer-based group support systems can organize and structure groups in 
a way that helps members use time more efficiently, focus on tasks better, and 


produce results that are as effective as groups that meet face-to-face. 


The combination of these three elements resulted in a proposed educational 
tool that is a web-based computer application where medical students, working in 
small groups, can construct diagrams of a patient case collaboratively. The process of 
collecting, recording and organizing patient information in a diagram could ensure 
that all members of the group share the same information. With the information 
displayed in a visual format, members could process the data through analysis, inter- 
pretation, evaluation and reflection more efficiently. 

A web-based format would provide members the flexibility to manage their 
schedule to accomplish their own work with the diagram on a computer at any time 
in any location. Likewise, instructors could monitor a group’s progress at their own 
convenience. Computer-based collaborative diagram construction could reduce the 
amount of in-class time required to discuss a case, it could help group members stay 
focussed on the task, and it would make members accountable for their contributions. 


A research question was established to focus future activities: 


How can diagrams serve as a tool to help medical students accomplish tasks 
associated with problem solving, critical thinking and decision making in 


small groups? 


EXECUTION 

The development of a web application is time consuming, expensive and it was 
beyond the scope of this thesis. Instead, a paper prototype that emulated the activi- 
ties of a web site was developed. In consultation with instructors in the Faculty of 
Medicine at the University of Alberta, a simulated patient case was selected for the 


‘nstruction, the form of the diagram was explored and interfaces were developed. 
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* Patient information 
A simulated patient case was selected and the relevance of each piece of information 


was determined with the assistance of faculty members in the Faculty of Medicine. 


* Form of the diagram 
After studying different configurations of diagrams, networks, maps and symbols, a 
network diagram construction was selected as the most appropriate. Network dia- 
grams are one of the few constructions that allow the recording, analyzing and inter- 
preting of the qualitative information typically found ina patient case. 

Size, colour, shape, value and gestalt psychology were among the design prin- 
ciples that were explored in the development of a set of notations to produce a 


construction that was readable and compact. 


* Interface sketches 
A series of interfaces were sketched to explore the types of interactions a user must 


perform on a computer to construct a diagram. 


PRODUCTION 

Based on the products of the execution stage, a diagram construction toolkit was 
developed. The kit contained three different sized nodes, each with three colours that 
corresponded to the three main routines involved in clinical reasoning: patient his- 
tory, physical exam and laboratory tests. A matrix for students to enter their differen- 
tial diagnosis was produced. A series of stickers to assign responsibility of an activity 
to a group member or label information with priority and urgency was created. A 
series of toolkits were produced as testing materials to provide to groups so they 


could construct diagrams as part of the evaluation stage. 


EVALUATION 

Four instructors and 13 students participated in a study. The instructors worked 

in pairs, while the students worked in groups of four or five. Each group created a 
diagram of a patient case using the diagram construction toolkit for 45 minutes. 
Afterward they completed a questionnaire that gauged their response to the activity 
and a focus discussion was conducted to provide students with the opportunity to 
elaborate on their answers. 

Data was collected using audio and video recordings, questionnaires, notes taken 
by observers, and the products of the diagram construction activity. The materials 
were analysed to assess: the performance of the paper prototype in facilitating stu- 
dents and instructors to create diagrams; their opinion of the exercise for problem- 
based learning tutorials; and the degree to which the diagrams showed signs of the 
tasks associated with higher-order thinking. 


Based on the results of the analysis, recommendations were made that will influ- 


ence the final design of the website and the definition of the needs for future research. 


Background research 


From the earliest stages of training, the purpose of medical education is to teach 
students the skills required to evaluate and identify an illness and prescribe a course 
of treatment. When students become doctors, it is their responsibility to pay due 
care and attention when a patient enters their offices complaining of an illness. A 
disease can have far reaching effects if it is not attended to properly. Therefore, a 
doctor must possess expert skills at evaluating a patient’s illness, identifying symp- 
toms through reasoning and research, and making a correct diagnosis of the medical 
problem (Barrows & Tamblyn, 1980). 

Traditional medical education is comprised of students attending lecture and 
laboratory in early years to develop an understanding of the basic sciences relevant 
to medical practice, such as: anatomy, physiology, microbiology, and immunol- 
ogy. In later years, students learn clinical skills through contact with patients. Some 
medical educators argue that the traditional approach is insufficient to deal with the 
complexity of medical science today and cannot teach the advanced problem solv- 
ing and decision making skills that are so critical to patient care. In medical schools 
around the world, problem-based learning has been incorporated with or completely 


replaced traditional medical instruction. 


PROBLEM-BASED LEARNING 


Problem-based learning, also known as PBL, is a method of instruction that enhanc- 
es problem solving and decision making skills by having students concentrate on a 
life-like problem in self-directed groups under the guidance of a tutor (Barrows & 
Tamblyn, 1980). While working through a clinical scenario, students attempt to un- 
derstand a patient’s problem while acquiring an integrated body of knowledge and 
developing clinical reasoning skills (Barrows & Tamblyn, 1980). New knowledge that 
is acquired and retained is structured in a way that it is easier to recall when a stu- 
dent is confronted with a similar problem in the future (Norman & Schmidt, 1992). 
In a typical problem-based learning tutorial, students are provided with a one- 
page brief of a simulated patient case that outlines the facts of the illness, patient 
history, physical exam and any pertinent laboratory results. Students are expected 
to identify which information is important or relevant, analyse the patient history 


to develop a differential diagnosis, and use physical exam and laboratory results to 
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rule in or rule out suspected diseases. In a PBL context, students are provided with a 
high degree of responsibility for their own learning. They are considered adults with 
a wealth of experience, education and interests (Barrows, 1988). If group members 
do not possess the knowledge required to address a particular component of a prob- 
lem, it is up to them to assume the responsibility for researching missing pieces. The 
tutor’s role is to facilitate and guide the process (Barrows, 1988). Depending on the 
complexity of the case, this process can be as short as a three-hour session or it can 


elapse over a period of days. 


CONSTRUCTIVISM 

The tenets of problem-based learning find support in the principles of constructivist 
learning. Constructivism is grounded in the notion that learning can be more pow- 
erful when students construct their own understanding of what they are studying, 
rather than having information presented to them (Eggen & Kauchak, 2004; Henson 
& Eller, 1999; Lefrancois, 2000). In a constructivist context, learning is facilitated 
through social interaction, and meaningful learning occurs when it is paired with an 
authentic task (Eggen & Kauchak, 2004; Henson & Eller, 1999; Lefrangois, 2000). Ed- 
ucational activities focussing on authentic tasks encourage students to think, reason 
and solve problems that they may encounter in daily activities (Eggen & Kauchak, 
2004; Henson & Eller, 1999). When students interact with one another they can 
share ideas, articulate their thinking and incorporate other perspectives with their 
own (Eggen & Kauchak, 2004; Lefrancois, 2000) (Ficure 3). 


GROUP PROBLEM-SOLVING 
Self-directed, group learning becomes a critical aspect of the learning opportunities 
that occur in PBL tutorials. In a group learning environment, members come with 
their own set of perspectives and biases, which create an atmosphere where a variety 
of viewpoints can be expressed. Diverse viewpoints, expressed through discussion, 
provide members of the group with a greater understanding of the complexities of 
the problem and can serve to generate ideas that are more creative, varied and numer- 
ous than an individual can hope to achieve on their own (Barker et al., 2001). Stu- 
dents benefit from the process of the group which helps them build skills to identify 
learning issues, brainstorm, test hypotheses, solve problems and make decisions (Bar- 
rows, 1988; Barrows & Tamblyn, 1980; Hak & Maguire, 2000). Studies have shown 
that groups engaging in problem solving activities produce results that contain fewer 
errors than individuals could generate working on their own (Barker et al., 2001). 

As a consequence of the autonomous, contextual and social nature of the learn- 


ing experience provided by PBL, students learn to be better problem solvers, better 
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Ficure 3: Effects of discussion on group understanding 


Before discussion After discussion 


unknown 
information 


information shared by @ information shared by information shared by 
two members of a group three members of a group four members of a group 


Small group interactions help individuals broaden their undertanding of a complex problem. 

The large outer circles represent all that could be known and understood about a problem. Each 
member of a group comes with their own experience and knowledge base. The overlapping inner 
circles represent each unique individual. Some members already share common knowledge 
base with other members in the group and, to a lesser degree, with everyone in the group. When 
members gather, share and interpret information in order to solve a problem they reach a 
mutual understanding. 


self-directed learners, are better able at integrating scientific knowledge in clinical 
situations and are highly motivated about the subject they are studying (Barrows & 
Tamblyn, 1980; Norman & Schmidt, 1992). 


HIGHER-ORDER THINKING 
The purpose of the teaching methods employed in PBL tutorials is to help students 
develop higher-order thinking skills. Higher-order thinking is the encompassing 
term used to describe the skills that require complex cognitive operations such as 
problem solving, critical thinking and decision making (Wakefield, 1996). It is dif- 
ficult to define each of these as independent operations since each contains activities 
that overlap and converge with one another. In the 1950s, a taxonomy of educational 
objectives was developed to help instructors identify and assess these higher-order 
thinking skills (Bloom et al., 1994) (Ficure 4). The taxonomy characterizes the de- 
velopment of skills as a continuum, but is also indicative of the thinking process an 
individual goes through to solve a problem. 

The most basic units of the taxonomy—knowledge, comprehension, applica- 


tion—address an individual’s ability to possess knowledge, comprehend it and apply 
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FicurE 4: Bloom’s taxonomy of educational objectives 


Lower-order thinking 


Source: Wakefield, 1996, p. 409 


The Taxonomy of Educational Objectives was created by the American National Board of Examin- 
ers in 1956 in an effort to classify educational behaviours for college and university examiners. 

It shows a continuum of progression from general skills to more specific behaviours. Each stage 
builds upon the foundations created by the previous one. Early educational training focuses on the 
first three, while higher education attempts to train individuals with skills in the following three. 

in the 50 years that have followed, the taxonomy has become a basic reference for educators. 


it to a general problem. Lower-order thinking is the term used to describe this basic 
set of operations where an individual is provided with a problem that can be solved 
by recalling information they possess. It is important for an individual to develop 
this set of skills so they can go on to solve more advanced problems. Higher-order 
thinking is used to describe a situation where the solution to a complex problem 
may reside outside the skills of an individual to recall and apply information they 
possess. Higher-order thinking is a set of skills that allow a person to strategically 
acquire knowledge to find a solution to a problem (Wakefield, 1996). At this more 
advanced stage of problem-solving, an individual must be able to analyse, synthesize 
and evaluate knowledge within the confines of a new problem. Critical thinking, 
reasoning, problem solving and decision making are all terms that are frequently 
used to describe activities and process that occur within the domain of higher- 
order thinking. 

A patient with a medical condition that requires treatment is an example of a 
complex problem that requires physicians to apply higher-order thinking. Conse- 
quently, medical schools are organized to incorporate a variety of teaching methods 
to help students develop the advanced problem-solving abilities required to diagnose 
a patient’s medical condition. Small groups and problem-based learning tutorials 
provide an opportunity for students to develop clinical reasoning and problem solv- 
ing skills by providing them the opportunity to integrate their knowledge of basic 


science with clinical skills by applying them to a patient case. 
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FRAMEWORK TO APPROACH CLINICAL CASES 
To assist the physician to handle the complexity of a patient case, a framework has 
been developed and refined over the years, which has become standard practice to 
approach patient cases (Bickley & Szilagyi, 2003). Students are taught this frame- 
work early on in medical school to encourage them to go through a case methodi- 
cally and ensure the proper information is collected from the patient. Students are 
expected to apply the framework in the following order: 

1. The patient is interviewed to collect a health history relevant to their 


chief complaint; 
2. A list of diagnostic hypotheses, called a differential diagnosis, is developed; 


3. A physical exam is performed to collect more information to refine the 


differential diagnosis list; 
4. Laboratory and diagnostic tests are ordered to confirm the hypothesis; and 


5. A treatment plan is developed appropriate to the patient’s needs. 


PBL: THEORY VERSUS PRACTICE 


From a theoretical perspective, problem-based learning should help students de- 
velop clinical reasoning and problem solving skills, however, this is not always the 
case when it is put into practice. Studies comparing the knowledge base and clinical 
performance of students trained in a PBL environment with the abilities of students 
from a traditional format have not identified significant differences (Colliver, 2000). 
Proponents of PBL argued that not enough research has been conducted into 
the activities and processes that occur in small group learning that are connected to 
the development of reasoning and problem solving skills (Albanese & Mitchell, 1993; 
Hak & Maguire, 2000). They argued that problem-based learning itself may not be 
the issue, but the problem may reside in how PBL tutorials are conducted. Tutors 
must be skilled at guiding and facilitating student learning and need to pay more at- 
tention to the group’s reasoning and problem solving processes (Patel et al., 1991). 
Studies have also shown that students immersed in PBL frequently employ back- 
ward reasoning—a tendency that is typical of novices—when they approach a prob- 
lem (Albanese & Mitchell, 1993). Backward reasoning is a method of solving a prob- 
lem where an individual starts with a possible solution and finds evidence to support 
the conclusion (Gilhooly, 1990). Experts, on the other hand, use the evidence they 
have collected to support possible conclusions, a method that is often called forward 


reasoning (Norman & Schmidt, 1992). While students who employ backward 


reasoning tend to produce more elaborate explanations using relevant data, they also 
tend to be more prone to errors. Without proper guidance from the tutor to correct 
erroneous reasoning, the development of expert clinical reasoning and problem 


solving skills may be impeded over time (Patel et al., 1991). 


PBL aT THE UNIVERSITY OF ALBERTA 

In 1998, the Faculty of Medicine and Dentistry at the University of Alberta intro- 
duced a new curriculum which was intended to increase the amount of autonomous, 
student-centered learning. The first two years of the curriculum are divided into 
nine systems-based teaching blocks, plus two long courses that deal with patient- 
centred care issues. This new curricular approach adopted by the Faculty could be 
described as a hybrid between a traditional medical school curriculum and a prob- 
lem-based learning curriculum. Lectures and laboratories provide students with an 
understanding of basic science, while problem-based and small group learning are 
among the methods used to encourage students to integrate knowledge with practice 
using clinical scenarios. 

The Faculty of Medicine and Dentistry at the University of Alberta adopted a 
modified curriculum in 1998, with one of its stated aims being the introduction 
of active learning strategies. Problem-based learning was introduced as one active 
learning strategy, and the method has gained wide acceptance in the curricula of 
many medical schools and by the North American medical school accrediting bod- 
ies. In the University of Alberta system, block directors have considerable autonomy 
in formulating the mix of learning methods which are used in each of the blocks. Ac- 
cording to Dr. David Rayner,' finding enough tutors is one of the biggest limitations 
to incorporating more PBL into the curriculum (personal communication, October 
14, 2003). 

At the present time there are approximately 160 medical students in each of the 
first two years of medical school at the University of Alberta. Small group and prob- 
lem-based learning sessions typically consist of 12 students. At least 13 tutors are 
required to accommodate PBL in a single block. If every block uses their own set 
of tutors to facilitate small group and problem-based learning, it would require the 
participation of a total of 120 physicians. 

Two instructors, who are also teaching block directors, in the Faculty of Medi- 
cine and Dentistry at the University of Alberta were interviewed during the early 


stages of investigation. Both were aware of the value of PBL as an instructional 


Dr. David Rayner is the Associate Dean of Undergraduate Medical Education, the Director of 


the Introductory block, an Associate Professor, and a Pathologist in the Faculty of Medicine. 
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method, but only one of them was using PBL tutorials in their teaching block. 

The instructor using PBL in their teaching block, indicated that a lack of tutors 
was undermining the ability to conduct tutorials. In an attempt to rectify the prob- 
lem, senior medical students were brought in to tutor PBL groups with unsatisfacto- 
ry results (personal communication, October 20, 2003). The second instructor who 
was not using PBL, was more concerned about the time required to cover a patient 
case. This instructor preferred to schedule the already limited contact time available 
more effectively (personal communication, May 7, 2004). A simple PBL case requires 
at least three hours to complete. 

Student opinions of problem-based learning tutorials were mixed. In a pre- 
liminary interview, two second-year medical students said that although they enjoy 
the opportunity to work through a patient case and apply the knowledge they have 
gained through laboratory and lecture, they have a number of complaints about 
PBL. Their biggest concerns were focussed on tutors who lecture, peers who shout 
out answers without thinking, and individuals who direct the conversation off topic 
(personal communication, April 30, 2004). They also said it was unfair that there are 
students who routinely did not participate in discussions and still receive a passing 
grade for their PBL work. They said they would like to see more accountability from 


students and instructors to ensure the proper functioning of PBL groups. 


COMPUTER-BASED GROUP SUPPORT SYSTEMS 


The issues raised by instructors and medical students regarding the hazards of au- 
tonomous learning are not unique to small group problem-based learning tutori- 
als. Individuals who work in teams in a business setting often complain about the 
frequency and duration of meetings, especially when members consistently stray off 
topic. Computer-based group support systems have been developed over the years to 
respond to these problems. 

Group support systems are computer applications that have been developed 
to help groups generate ideas, evaluate options and make decisions (Galanes et al., 
2004). Specialized systems allow groups to set agendas, brainstorm, organize and 
gather detailed information, vote on alternatives, collect instant feedback and pro- 
vide a means of disseminating results (Ficure 5). Group support systems are a term 
also used to describe ubiquitous technological options such as discussion forums, 
e-mail, digital workspaces, instant messaging and video conferencing (Cragan et al., 
2004). These communication tools can be used to form virtual groups that allow in- 
dividuals who reside a great distance from one another to work together. Conversely, 


group support systems can be used by people who may live close to one another, but 
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FicurE 5: GroupSystems’ GS II team decision making and collaborative thinking tool 
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Screenshots from GroupSystems’ website indicated that GS Il is a highly structured and heavily 
text based decision making and collaborative thinking environment. This type of approach encour- 
ages a very linear and sequential way of thinking that may not always be good for a complex prob- 
lem. Using the software, teams can: (1) set an agenda for work; (2) brainstorm new ideas; 

(3) organize the brainstorming concepts; (4) describe those concepts in more detail; (5) 

create a tree structure with a logical flow; (6) weigh each option; (7) tabulate the results; and 

(8) generate a report. 


face demanding schedules that reduce their opportunities to meet face-to-face. 

Studies into group decision making work facilitated by computer technology 
have shown that results are as good as those accomplished by groups interacting in 
a face-to-face environment (Galanes et al., 2004). Also, group members taking ad- 
vantage of computer-based group support systems are more satisfied because of the 
freedom and flexibility the tools provide. For example, discussions through the com- 
puter can occur when the group agrees to meet ata pre-arranged time, but contribu- 
tions can also be made during off-hours. People using group support systems have 
reported that they also seem to be able to focus on tasks better and spend less time 
in meetings (Galanes et al., 2004). The computer technology that delivers these sys- 
tems allows groups to keep long-term records of their activities, which increase 
accountability among members (Galanes et al., 2004). 

Still, the advantages to group communication offered by computer-based group 
support systems cannot entirely replace face-to-face meetings. Complex commu- 
nication tasks such as those that require coordination, conflict, or negotiation may 


suffer as a result of the loss of the ability to detect verbal or visual cues (Cragan et al., 
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2004). In addition, the communication that occurs using computer support systems 
tends to be text dominant due to the nature of the medium. Although these systems 
often provide a record of the discussion, the findings of the group, a timeline for 
project completion and a report of the activities of the group, these documents dis- 
play information in a purely text-based format. The structural configuration of writ- 
ten text encourages a linear and sequential way of thinking about a problem, which 
increases the potential to overlook important elements and denies the opportunity 


to recognize relationships within a set of data (Frascara, 2001). 


VISUAL REPRESENTATIONS 


Visual representations, in the form of diagrams, have an advantage over aural or text 
formats because they allow “hierarchies, inclusions, simultaneity, distinction of levels, 
multiplicity of kinds and complexity of connections” that encourage the contextualiza- 
tion and integration of information into meaningful wholes (Frascara, 2001, p. 166). 

Visual representations help the viewer to analyse, interpret and understand in- 
formation and the complex network of interactions that can occur within. When in- 
formation becomes more abstract and complex, visual representations can help cope 
with complexity. 

In their article featuring diagrams as tools for worldmaking, Helmer Poggenpohl 
and Winkler (1992) outlined J.J. Gibson’s theories on human perception as relevant 
to the study of visual representations. Gibson suggested that humans are tuned to 
pick up changes in an environment that is generally persistent (Helmer Poggenpohl 
& Winkler, 1992). Humans are “hard-wired” with the ability to survey a panorama 
and pay attention to the observable events that disrupt structures that remain con- 
stant (Helmer Poggenpohl & Winkler, 1992). Likewise, human perception is tuned to 
perceive visual representations in a manner similar to the way we observe the envi- 
ronment that surrounds us (Helmer Poggenpohl & Winkler, 1992). 

The specialization of the human perception system provides individuals with 
an opportunity to take readings at various levels of detail which allows them to see 
relationships, make comparisons and sort through elements in order to understand 
complex data (Tufte, 1991). Tufte (1991) defined two general levels of information: 
micro and macro (p. 37). The macroscopic view permits an individual to scan the 
whole to get a sense of what the information is about and notice areas within the 
data that require more attention. The microscopic level allows an individual to 
analyse specific information in detail. 

In a departure from Tufte’s two levels of information, Bertin distinguished three 


general levels of information: overall, intermediate and elementary (Bertin, 1983). 


16 


mae. aaa 
a) ee. 7. — 


The overall level is the entire set of data that is displayed to construct the entire 
picture. Bertin’s description of the overall level is the equivalent of Tufte’s descrip- 
tion of the macroscopic view. The elementary level of Bertin’s classification system is 
formed by the specific individual pieces of data that combine together to comprise 
the whole picture. This level of detail roughly corresponds to Tufte’s description of 
the microscopic view. 

Where Bertin departs from Tufte is in his addition of the intermediate level of 
information. In a complex set of data, it is impossible to retain each individual piece 
of data, so a viewer attempts to find similar elements to group and classify informa- 
tion. This intermediate level is used to describe all the subsets of data that are be- 
tween the two extremes of overall and elementary. According to Bertin, the interme- 
diate level allows us to analyse relationships within the overall set of data, while the 
data contained at the elementary level allows us to understand and decide (Figure 6). 

Bertin’s description of the levels of information is similar to Tufte’s, but it is 
more precise since it includes a level of differentiation that is important to the task of 
information processing. If one were to analyse a tree using Tufte’s description, only 
the tree and the leaves would be apparent. The tree corresponds to the macroscopic 
level, while the leaves correspond to the microscopic level. According to Bertin’s defi- 
nition, one can see that a tree is comprised of a series of branches that contain many 
leaves. In this reading the tree corresponds to the overall, the branches represent the 


intermediate level, and the leaves are the elementary level. 
Ficure 6: Levels of reading: Overall, intermediate and elementary 


Source: Bertin, 1983, p.141 


The first diagram represents the elementary level of reading. On one particular day, the stock price 
was at one particular level. The second diagram shows the intermediate level, where the analysis 
can be expanded. Over the first three days, the stock fell. At the overall level, one can expand the 
search even wider to answer the much larger question, “What was the overall trend of the stock?” 
The stock rose. 
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A SYSTEM FOR NOTATION AND ORGANIZATION 

Through a series of graphic notations on a page or on a computer screen, people can 
record information and share it with others. An individual reading a visual represen- 
tation can scan the whole to get a sense of the overall terrain, compare and contrast 
groups of information and focus on specific details to gain an understanding of the 
data. Through this process of understanding the information contained within a 
visual representation, individuals or groups can efficiently solve problems and make 
informed decisions (Bertin, 1981, 1983; Tufte, 1991). 

A number of graphic techniques and principles of visual organization can be 
combined to create a display on a surface that eases the process of viewing and 
interpreting complex information. Jacques Bertin defined these as: impositions, 
implantations, and visual variables. 

According to Bertin (1983), impositions refer to the utilization of the planar 
dimensions, which can be divided into four groups: diagrams, networks, maps and 
symbols. Regarding diagrams and networks, in particular, the use of the plane can 
vary depending on the number of components that need to be represented 
(Ficure 7). This further subdivision is referred to as the type of imposition. 

Implantations are the rudimentary figures of plane geometry—point, line, and 
area—that are used to mark or notate the location of components on the plane 
(Bertin, 1983). Size, value, texture, colour, orientation and shape are the six visual 
variables that a designer can use to distinguish implantations (Bertin, 1983) 

(Ficure 8). Visual variables can be combined in any fashion to influence how the 
reader distinguishes and perceives data by indicating similarity, difference, order or 
proportion (Bertin, 1983). Not every variable is suitable for representing every com- 
ponent, since the human perceptual system has its limitations. For example, small 
differences in scale, value and colour can be difficult to perceive—even when com- 
pared adjacent to one another. Also, some textures can produce a moiré effect that 
can be distracting for readers (Bertin, 1981; Tufte, 1984). 

Implantations and visual variables represent the basic visual characteristics and 
elements of a graphic system. Position, direction and space are the design principles 
that influence visual interactions between each of these components (Bowers, 1998). 

Position is the location of an element in relation to another element that can be 
used to create points of focus or tension in a composition (Bowers, 1998). In basic 
terms, objects can be distant, overlapping or touching. A suggestion of movement 
can be made when an object is cropped by the edge of a frame. Direction refers to 
the horizontal, vertical or diagonal arrangement of components that leads our eye to 


follow their direction (Bowers, 1998). These components can be positioned parallel 
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or perpendicular to one another to suggest motion or draw attention to a focal point. 
Space is the area around and between elements that can be used by the designer to 
group, separate, and emphasize elements to better distinguish between elements and 
their roles in the overall composition (Bowers, 1998). White space or figure/ground 
relationship are terms that are also often used to describe this interaction. 

Through the use of organizational principles that control the interaction of ele- 
ments ona page a designer can control elements to convey an intended meaning to the 
viewer. Depth, perspective, weight, symmetry, hierarchies, contrast, rhythm and pro- 
portion are all organizational means designers have at their disposal. The appearance, 
interaction and relation of elements are important for communication, but the way in 


which they are perceived by the viewer influences how the message is interpreted. 
FIGURE 7: Groups of imposition and types of imposition 


TYPES OF IMPOSITION 


Arrangement Rectilinear Circular Orthogonal Polar 


Diagrams 


GROUPS OF IMPOSITION 


Networks 


Maps 
sy Source: Bertin, 1983, p. 52 
The utilization of two planar dimensions is called “imposi- 
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tions”. Groups of imposition can be classified according to 
the way in which components are represented. These groups 
consist of diagrams, networks, maps and symbols. The table 
figure also shows that the components can be inscribed in 

a variety of ways for diagrams and networks. These various 
ways of inscription are called types of imposition. 


Symbols 
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Gestalt psychology explains the way humans perceive and organize visual com- 
ponents into meaningful wholes when the perception of each individual part differs 
(Bowers, 1998). The principles of gestalt are divided into three major parts: closure, 
proximity and similarity (Ficure 9). Closure is a principle in which individual ele- 
ments are located in relation to one another so the form is perceived as a whole, 
rather than distinct elements. Proximity is a principle in which relationships are 
suggested by how close an object is to another. Similarity is a principle in which the 
attributes of an object—for example, size or shape—help link figures together. In the 
context of visual representations, each of these principles can be combined to orga- 
nize elements on a surface in various ways that influence the way information is read 


and interpreted by the viewer. 


Figure 8: /mplantations and visual variables 


Implantations Visual variables 
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Source; Bertin, 1981, p. 187 


Implantations are the three elementary figures of plane geometry that are used to assign a 
mark on the plane. Visual variables are the graphic notations that can be applied to each of the 
implantations to distinguish relationships. 


Figure 9: Gestalt principles: Closure, proximity and similarity 


Source: Bowers, 1998 


Gestalt psychology is the way humans perceive and organize visual components into meaningful 
wholes when the meaning of each individual part differs. The three basic gestalt principles are: (A) 
closure, (B) proximity, and (C) similarity. 


PERCEPTION AND COGNITION 


To this point, this review of the research has centred on the formal aspects that facili- 
tate the process of a person perceiving information contained within a visual repre- 
sentation. Although we may perceive something, we may not necessarily make sense 
of it. Perception and cognition are related, but independent of one other. Perception 
is the end result of a process where a representation of an object is temporarily stored 
in memory, but has not yet been assigned any meaning by the mind (Spence, 2001). 
Cognition is the process by which meaning is assigned to a representation when an 
internal model is created or modified and established in memory (Spence, 2001). 
When a person is attempting to deal with a complex set of information, visual 
representations can serve as a device to facilitate the transition from perception to 
cognition. Bertin (1983) describes this function of visual representations as “arti- 
ficial graphic memory”. In the cognitive sciences, artificial graphic memory is also 
called an external representation (Spence, 2001). Human short-term memory has a 
very limited storage and processing capacity with short-term memory only capable 
of retaining seven units of information, plus or minus two (Bertin, 1983; Spence, 
2001). Beyond that number, the margin of error in the ability to recall data begins 
to increase significantly. Chunking information in groups of three may help, but a 
person will only be able to accurately remember seven chunks (Spence, 2001). When 
it comes to highly complex problems, involving more units of information than can 
be handled by short-term memory, visual representations can store information in 
a way that allows readers to analyse and process the information efficiently without 


taxing their memories. 
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MENTAL MODELS 

An external artefact, such as a visual representation, can serve as a reference for an 
individual to help construct an internal or mental model of a process that is very 
complex. A new mental model is constructed when individuals examine new infor- 
mation and integrate it with the models they already possess. The new mental model 
allows an individual to process and interpret new information within the construct 
(Spence, 2001). The quality of the interpretation of the internal model is contingent 
upon the quality of existing mental models and the quality of the visual representa- 
tion. The quality of a visual representation is contingent upon the creation of the 
appropriate structural and functional components by the designer to communicate 


information to the reader. 


VISUAL REPRESENTATIONS AS A LEARNING RESOURCE 

The possibility of visual representations to assist in the display, analysis, and inter- 
pretation of information offers a powerful pedagogical resource that can be used by 
students in activities that require finding solutions to complex problems (Frascara, 
2001; Simms-Knight, 1992). Simms-Knight contended that visual representations can 
aid in learning and problem solving when: (a) they allow students to search through 
complex information; (b) they make information that is implicit in a text explicit 
through visuals; and (c) they organize and integrate complex information (Simms- 
Knight, 1992). Visual representations can structure problems into groups in a way 
that represent the essential components that are required to help find a solution. 
Visuals help students to create a conceptual model, see relationships, and integrate 
information they already know into that model. All of these operations can be done 


simultaneously, which is something text alone cannot achieve (Simms-Knight, 1992). 


MIND MAPS AND CONCEPT MAPS 

Mind maps and concept maps are two strategies used in elementary and second- 
ary schools to visually organize knowledge to aid comprehension (Wakefield, 1996; 
Ware, 2000). When mind maps are constructed as a group exercise, they can help 
build common understanding among group members (Ware, 2000). 

Mind maps were developed by Tony Buzan in the 1960s as a method of helping an 
individual think clearer and learn better (Lefran¢ois, 2000). The intention is for a per- 
son to combine single words, numbers, images, and colour using spatial arrangement 
to organize issues relating to a central theme. They can be problematic for another 
person reading it because single words describing points cannot provide the full con- 
text of what is being stated. Also, without a convention, mind maps can become a free- 


form expression of a theme that is only useful to the person who created it (Ficure 10). 


Faia, ne oodd eb anemn 
i ne Saree = 
cae i: | ae =f 


mig os abt ane 


FIGURE 10: Mind maps 


http://www.mind-map.com/EN/mindmaps/gallery.html 


Mind mapping is a technique developed to map issues around a central theme while taking advan- 
tage of the “full range of cortical skills” —words, numbers, images, and colour using spatial ar- 
rangement. The laws of mind mapping encourage people to develop a personal style that is mean- 
ingful to themselves. Unfortunately, as expressed in the large map above, these freeform 
expressions can be difficult for others to understand. 


FIGURE 11: Concept maps 
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Concept maps are a means of spatially organizing information. This mapping technique starts 
with a central concept and uses linking words to bridge concepts that are subordinately related. 
Information is structured in a hierarchical arrangement that forces the viewer to read in a linear 
and sequential way. If we are to examine the region of the map charted by numbers 1 through 6, it 
would read like this: “Concept maps represent organized knowledge necessary for effective teach- 
ing and effective learning”. This approach violates the principles of levels of reading and reduces 
the possibilities to see inclusions and simultaneity. 
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Concept mapping is a technique, similar to mind mapping, that was devel- 
oped by Joseph Novak in the 1960s. The major difference between the two is that 
the construction of concept maps must conform to a certain convention. Concept 
maps start with a central concept and use linking words to bridge concepts that are 
subordinately related (Ware, 2000). As a consequence of the formal aspects of the 
construction, there is a top down hierarchy that is implied. The reading of such a 


diagram, while faster than a text, is still linear and sequential (Ficure 11). 


Analysis and synthesis 


The background research suggests that problem-based learning has theoretical ad- 
vantages as a teaching method for medical students, however its implementation 
presents several problems. A computer-based group support system could provide 
structure and organization to the activities of small groups participating in problem- 
based learning tutorials in a way that would conform to the spirit and intent of PBL 


while addressing issues raised by students and instructors such as: 


- shortage of tutors to facilitate PBL tutorials; 
* opportunities for autonomous student learning; 
- effective use of face-to-face meeting time; and 


* student accountability for participation in PBL tutorials. 

Existing computer-based group support systems rely heavily on text laden forms 
of communication. In the context of medical students, current systems would pro- 
mote a linear and sequential way of thinking about patient cases which would not 
be able to support novices in the development of reasoning through and making a 
diagnosis for cases containing a complex array of information. 

Instead, a system that encourages groups of students to work together to cre- 
ate visual representations as a method of solving a patient case could provide an 
organizational structure and facilitate the cognitive operations required by allowing 
students to: 


- collect and record patient information; 
- organize data ona surface using visual variables such as colour, size and position; 


communicate the same information to members of the group to impart mutual 


understanding; 
- analyse and interpret information within the context of the problem; and 


- evaluate and reflect upon the results. 


24 


Diagrams constructed collaboratively on a computer by groups of students 
could also allow instructors to examine the group’s problem-solving process and 
critique their work. 


PROBLEM DEFINITION 


The purpose of this project is to conceive, create and evaluate an educational re- 
source that helps groups of medical students to engage in higher-order thinking 
activities to diagnose a patient’s illness. 

The development of the educational resource is founded upon three premises 
arising from the background research: 


1. Problem-based learning tutorials enable medical students to develop higher- 
order thinking skills by providing an opportunity to work on an authentic task 


as part of an autonomous small group. 


2. Visual representations can organize and display complex information in a way 
that helps individuals efficiently identify, analyse, interpret and evaluate to 
diagnose a problem. 


3. Computer-based group support systems can organize and structure groups in 
a way that helps members use time more efficiently, focus on tasks better, and 
produce results that are as effective as groups that meet face-to-face. 

Upon the foundation of these the premises, an educational resource that pro- 
vides groups of medical students with the tools necessary to construct diagrams of 
patient cases is proposed. Such a resource will conform to the following objectives. 


The educational resource will allow students to: 


1, Visually organize and display information in a way that 1s beneficial 
to the identification, analysis, interpretation and evaluation of data to 


reach a diagnosis. 


2. Share their own viewpoints through discussion, identify learning issues, 
brainstorm, test hypotheses, and diagnose an authentic problem in a 


small group, autonomous learning environment. 


3. Structure time more efficiently, focus on tasks better, and produce effective 


problem-solving results. 


While the approach proposed outlined in the objectives finds support in the 
background research, in keeping with user-centred design practices, the response of 
the audience will be gauged. The following research question was proposed to help 
shape the design response and determine if the educational tool will serve the needs 


of students and instructors and perform the functions for which it is intended. 
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How can diagrams serve as a tool to help medical students accomplish tasks 
associated with problem solving, critical thinking and decision making in 


small groups? 
This broad question was then broken down into general categories that address: 
1. The requirements of students constructing diagrams as a group; 


2. Students’ and instructors’ reactions to the group diagram 


construction activity; and 


3. The potential of diagrams to help develop tasks associated with 
higher-order thinking. 


More specific questions were defined for each of these general categories: 
REQUIREMENTS OF STUDENTS CONSTRUCTING DIAGRAMS 


Can a group of medical students, without formal visual training, use a 
set of tools to construct diagrams that outline their clinical-reasoning and 


problem solving process? 


What kind of tools are required to help medical students organize 
information in a diagram that can help them identify, analyse, interpret 


and evaluate information to diagnose a case? 


Are the tools created and provided in the diagram construction toolkit 


simple and effective for students to use? 


STUDENT AND INSTRUCTOR REACTIONS 
Do students enjoy the activity of creating diagrams of a patient case and do 


they find them beneficial for diagnosing a patient case? 


Do instructors find the tool beneficial for conducting small group, PBL 


sessions and would they encourage students to use it? 


Would students and instructors use a web-based educational resource to 


construct diagrams as a group? 


DEVELOPMENT OF TASKS ASSOCIATED WITH HIGHER-ORDER THINKING 


Can the products of the diagram construction activity be assessed in terms of 


the groups’ ability to identify, analyse, interpret and evaluate information? 


Can the analysis of the products of the diagram construction activity help 


instructors provide feedback to students? 
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Design response 


DESIGNING THE PROTOTYPE 


In order to assess the effectiveness of the educational resource, a fully functional 
computer-based prototype should be developed and tested with students and in- 
structors to answer the questions posed in this study. However, the production of 
such a prototype can be an expensive and time consuming task. Moreover, if the 
prototype fails to perform according to desired specifications, it can be difficult to 
distinguish between faults in the design that may influence the interaction between 
computers and the human operators or in the design of the activity itself. Paper 
prototypes have demonstrated their effectiveness at emulating specific operations a 
computer can perform to evaluate the way humans may interact with the proposed 
solutions (Snyder, 2003). The use of paper prototypes at the formative stages ofa 
project can help the researcher identify flaws in the design of the activity, indepen- 
dent of the user interface that is required for that activity to occur on the computer. 
Using paper prototypes at this stage can reduce production costs and the need to go 


through costly revisions and corrections at later stages of the process. 


PATIENT INFORMATION 


Before developing a paper prototype, it was essential to first understand the infor- 
mation contained within a patient case to analyse how students may construct a 
diagram of the case. A patient case was selected with help from a professor in the 
Faculty of Medicine and Dentistry at the University of Alberta with experience in 
tutoring problem-based learning. The relevant pieces of information in the case were 
identified and, with the assistance of medical experts, a rationale and a description 
were produced for each piece. 

Using markers, large sheets of paper and sticky notes, information was plotted 
to explore shapes and organizational strategies of the diagram to find an appropri- 
ate arrangement. After attempts to use a network diagram spread across columns 
and a diagram organized within three large circles (Ficure 12 & 13), an organization 
that clustered information around three central themes seemed most appropriate to 
display the amount of information the case contained since the organization corre- 


sponds to the approach used by physicians to reveal a patient’s signs and symptoms. 


27 


‘ abediadala ataadeell 


FiGure 12: Patient information summary exploration 
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Photos depicting early explorations to map patient information in a network diagram using paper 
and markers. These explorations helped determine the appropriate format of a diagram to contain 


large amounts of information. 


FIGURE 13: Patient information summary exploration: networks 
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A less structured network diagram, similar to a mind map, was deemed appropriate to plot a large 
amount of information. History, Physical Exam and Tests were selected as three large headings 
around which information could be mapped. The relative distance between the large headers and 
the individual pieces of information helped identify large groups. Lines could link pieces of infor- 
mation and colour could be used to highlight important data. 


FORM OF THE DIAGRAM 


A system of notation and organization that made most effective use of the prin- 
ciples of visual organization and was appropriate to the amount of information con- 
tained in a case needed to be selected. Diagrams, networks, maps and symbols were 


studied. Most of these visual representations are appropriate to quantitative data. 
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Considering the majority of information contained within a clinical case is quali- 
tative in nature, a network diagram was selected because it provided the qualities 
important to recording, analyzing and interpreting the information. A network is a 
form of diagram that affords opportunities to establish relationships among sim1- 
larly related elements on a plane and it can yield the most efficient information when 


it is transformed to discover meaningful relationships. 


DESIGN PRINCIPLES 


A system of notation using line, size, value, colour and shape was developed to pro- 
vide students with the tools necessary to construct their own diagrams. Since the 
diagrams will eventually appear on a computer screen, texture and orientation were 
removed from the list of visual variables since they can produce unwanted moiré ef- 
fects when they are combined with other variables on the screen. 

The text of the patient information was input into the computer and a series of 
diagrams were created. Based on Bertin’s levels of information and the limitations of 
short-term memory, three levels of distinction appeared to be sufficient. 

Three colours were selected to correspond to the three main routines involved in 
clinical reasoning: patient history, physical exam and laboratory tests. A medium blue, 
a medium green and a medium burgundy were selected because they were easy to rec- 
ognize as distinct and were pleasant to look at for extended periods of time. 

A series of explorations were conducted where colour was the standard visual 
variable used to distinguish the three major groups. Different combinations of shade, 
shape, size of shape and width of line were applied to each diagram of the series to 
test how the distinctiveness of the three levels of information contained within the 
diagram would vary (Figure 14). 

Through trial, error, analysis and evaluation, a system that used a graduated size 
of circle for each of the levels of importance was selected from the possibilities that 
were explored. This configuration was chosen because levels of information were 
easily distinguishable and it used space efficiently so it could fit on a computer screen 
without violating principles of good design. To further reduce clutter, lines were 
eliminated and a version was produced that used the position of the circles in space 


to suggest relationships (Ficure 15). 


INTERFACE SKETCHES 


Parallel to the diagram explorations, a series of interface sketches were developed to 


determine the kinds of interactions that were required to construct the diagrams, the 


Ficure 14: Exploration of visual variables 


After the patient information was transcribed into text on the computer, a series of explorations 
were made using visual variables. Colour was applied consistently as a means of distinguishing 
major groups of information. In an early version (1), a different size of circles was used for each lev- 
el of information. In a second version (2), shapes were used to help further distinguish groups and 
the proportionate size of a node grew depending on the number of sub-nodes. Ina third attempt 
(3), shapes were used to distinguish each level of information. A consistent shape was determined 
to be the most readable, so a fourth attempt (4) varied the size of the circle to indicate the amount 
of information each node contained. 
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Figure 15: Final variation of patient diagram 
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In the final variation of the diagram, efforts were made to find an economical display for the infor- 
mation that would make optimal use of the computer screen. Circles were selected because they 
provide the most space for adjacent nodes when compared to other shapes. Up to nine sub-nodes 
can encircle a single node (see inset). Lines linking pieces of information were removed to reduce 
clutter, Instead, relationships are shown by the combination of colour and position of nodes next to 
one another. 


tools that might be required and examine how they might be incorporated into 
a computer interface. 

The first set of prototypes embodied a system for creating nodes that featured 
a complex set of symbols. Plus and minus symbols allowed a user to add or delete 
nodes and a menu could allow the user to input information (Ficure 16). To simplify 
the system, another version was created where a user could make modifications to 
the node using a single menu that contained all the same functions (Ficure 17). The 
disadvantage of this type of interface design is that the system would have to deter- 
mine automatically where the nodes were positioned which may have an impact on 


the way the diagrams can be read. 
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FiGuRE 16: Interface sketch 1 
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Ficure 17: Interface Sketch 2 
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Ficure 19: /nterface Sketch 4 
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To develop a system that allowed for a more flexible method of diagramming, a 
second set of prototypes was created where the tools were located in a toolbox 
(Figure 18). The user could drag and drop the tools onto the diagram space in the po- 
sition they desired. In the toolbox, there are specialized tools for adding and deleting 
a group or individual nodes, converting a group to a single node or vice versa and a 
tool to link nodes. 

Additional tools were developed to provide users with the capability of tagging 
information so it could be searched and sorted later. Using these tools, the user could 
assign certain properties to the nodes such as responsibility, relevance and prior- 
ity. When a user needed to, they could make a query using the computer that would 
search and sort the tagged information to enhance the users’ ability to analyse and 
interpret a certain set of data. 

When a form for the diagram was finally established through the explorations 
mentioned previously, a fourth version of the interface was developed that incor- 
porated the new diagram and featured a reduced set of diagram construction tools 
(Figure 19). 

While more work needs to be done to develop the interface, the creation of these 
sketches was important to determine how the space could be used, how text could 
be incorporated with the diagram and features that could be added to enhance the 


functionality of the website concept. 


PAPER PROTOTYPES 


With the establishment of a visual system to display information in a diagram and 
some concepts for a website interface, the task of creating a paper prototype to test 
the overall concept was simplified. 

Using the framework for a clinical case as an example, three large nodes were 
created: patient history, physical exam, and diagnostic and laboratory. Three sets of 
smaller nodes of two different sizes were created that operated like envelopes. Each 
envelope/node had a space to write a title and place stickers. An insert was created 
that provided a space to write a rationale for the information (Figure 20), 

This configuration for nodes was developed to emulate the interface sketches. 
On each of the versions of the interface, there was a method to create and title a 
node. There was also a method to assign properties to the node and a space to write 
a rationale. To minimize the potential for clutter on the screen, the rationale was ac- 
cessed in a separate window. 

Prior to testing with students, the paper prototype was informally tried by 


graduate students in visual communication design, which helped to find out that the 
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FIGURE 20: Paper prototype, version 1 
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envelope and insert configuration, although similar to the website, was a time 
consuming operation on paper. 

The nodes were simplified so the amount of time to perform the activity was re- 
duced. The elements on the nodes and inserts in the previous version were combined 
onto the nodes. The title space on the nodes became smaller, the rationale space was 
moved from the insert to the node and the position of the stickers was moved to 
provide more space for the rationale. 

Stickers were created so information could be tagged to help the searchability 
and selectability of information. Priority, responsibility and relevance were deter- 
mined as potential categories to tag information. Stickers to indicate levels of prior- 
ity were created, six responsibility stickers were created for each member of a group, 


and two relevance stickers were created. 


Evaluation 


STUDY DESIGN 


The purpose of this study was to determine if medical students, with no formal 
visual training, can construct meaningful diagrams of a patient case when they are 
provided the appropriate tools, and if the constructions can demonstrate tasks asso- 


ciated with higher-order thinking. 
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PARTICIPANTS 
The study was conducted with the approval of the Arts, Science and Law Research 
Ethics Board at the University of Alberta (APPENDIX A). 

A total of 13 students and four instructors participated in the study. The stu- 
dents were divided into three groups and the instructors were divided into two 
groups. Instructors were separated from students to see if a difference in an 
expert’s ability with clinical practice and a novice’s ability would influence the 
diagram construction. 

To invite volunteers to participate, an e-mail was sent to the entire class of 160 
second-year medical students in advance of the testing dates. The researcher ap- 
peared before the second-year class to make an appeal for volunteers and the visit 
was followed with a second e-mail. 

An e-mail invitation was sent to a list of instructors who were selected for their 
possible interest in the approach being proposed for PBL tutorials. Those instructors 
who elected to participate had all attended a presentation on the topic of this thesis 
delivered by the researcher to the Division of Studies in Medical Education Journal 
Club on January 25, 2005. Two of the instructors had been consulted regularly by the 


researcher throughout the entire research process. 


MATERIALS 


DIAGRAM CONSTRUCTION TOOLKIT 

Participants were provided with a diagram construction toolkit that consisted of 
three colour-coded sets of nodes, each of them containing three sizes of nodes. Par- 
ticipants were also provided with stickers that could be used to indicate responsibility, 
urgency, and level of priority, and a differential diagnosis matrix. Masking tape, ball- 


point pens and erasable markers were also included in the kit (Ficure 21 & APPENDIX B). 


PATIENT CASE 

A patient case was selected with the advice from a member of the Faculty of Medi- 
cine and Dentistry with experience in tutoring problem-based learning. The case was 
obtained from a set of cases developed by the Northeastern Ohio University’s Faculty 
of Medicine. 

The case is a page-long description that outlines the chief complaints and basic 
patient history of a 33 year-old male showing signs of Jaundice (Aprenoix B). Physical 
exam and laboratory results—which would have made the solution obvious—were 
detailed on separate cards and retained by the researcher. The cards provided infor- 


mation on things such as abdominal exams, liver function tests, and complete blood 
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FiGuRE 21: Diagram construction toolkit 
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counts (Appendix B). During the testing, the cards were distributed by the researcher 
to the students when they asked for them. 


QUESTIONNAIRES 


Questionnaires were developed to ask participants about: 


- preferred learning methods; 
- the use of the framework to approach a patient case; 


* opinions of the diagram construction activity conducted as 


part of the study; and 


- their use of computers. 
The responses for each question were structured on a Likert scale and additional 
space was provided for comments. The phrasing of questions was modified slightly 
to be appropriate for the instructors and the students (ApPenpix B). 


FOCUS GROUP QUESTIONS 
Focus group questions, although similar to those posed in the survey questionnaires, 
were asked to elicit more elaborate responses from the participants and to facilitate a 


discussion about the approach and adjustments that could be made (Aprenoix B). 


ANIMATED COMPUTER DEMONSTRATION 
An animated mock-up of the prototype was shown to participants during the focus 
testing to demonstrate how a website could perform and to solicit ideas for modifi- 


cation and improvement (Ficure 22). 
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FIGURE 22: Animated computer demonstration 
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A series of animated screens demonstrated how the diagram construction could operate using 

a computer application. The screen is blank with three larges nodes at the beginning. Nodes are 
added until the whole diagram is complete. Properties assigned to nodes containing information is 
used to search and select items in the diagram. A side-by-side comparison could help students and 
instructors assess the work. 


aH 


INSTRUCTIONS 
Each participant received the same instructions on what to do during the testing and 
how to carry out the diagram construction activity. A summary sheet containing ob- 


jectives for the activity was provided to each participant (APPENDIX B). 


SETTING 


Each of the five study sessions occurred in rooms in the Medical Sciences Building, 
which is part of the Health Sciences complex at the University of Alberta. The three 
student sessions were conducted in one of the 12 rooms built specifically for small 
group and problem-based learning. These rooms are familiar and comfortable spaces 
that these students regularly use for PBL tutorials. The two instructor sessions was 


held in a meeting room in the Division of Anatomy offices. 


ROLE OF THE RESEARCHERS 


The researcher and an observer were present at each of the five group testing ses- 
sions. The researcher performed as the coordinator for the activity by: outlining the 
study to the participants; distributing and collecting permission forms; operating the 
camera and recording equipment; distributing the questionnaire; and conducting 
the focus groups. 

The instructions for the use of the tools for the diagram construction activity 
were broad and open-ended. Neither the researcher nor the observer answered any 
direct questions about the use of the tools during the diagram construction activity. 

The researcher monitored the activity and, when asked by the participants, he 
provided additional physical exam information and laboratory results necessary to 
solve the case. 

The observer’s responsibility was to record general impressions about the way 
participants were constructing the diagram, the participants’ behaviour towards each 


other, and other general comments. 


PROCEDURE 


A total of five groups participated in one and a half hour testing sessions during a 
week long period. Three groups of four to five medical students and two groups of 
two instructors were involved. Each group session began with an explanation of the 
study and participants were asked to fill out a consent form. Participants were then 
asked to participate in a diagram construction activity of a patient case, fill out a 


questionnaire and discuss the activity ina focus group. 
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The diagram construction activity was the central activity of the testing. Stu- 
dents and instructors were provided with a diagram construction toolkit and a brief 
patient case, and allowed 45 minutes to construct a diagram for the case. An identical 
diagram construction toolkit was provided to each of the groups. Groups of instruc- 
tors created diagrams separate from groups of students so the results could be anal- 
ysed for similarities and differences in the way experts and novices performed. 

For 20 minutes immediately following the conclusion of the diagram construc- 
tion activity, all the participants were asked to fill out a questionnaire. The question- 
naires were intended to collect responses about the activity and the prototype. The 
questionnaires also collected participants’ opinions of the importance of a frame- 
work to approach a patient case, preferences for learning methods and habits using 
the computer. The questionnaire was also intended to stimulate people’s thinking 
about the process to prime them for discussion during the focus group discussion. 

For 30 minutes following the diagram construction activity and completion of 
the questionnaire, participants were asked to discuss the activity and express their 
point-of-view in a more detailed way than accomplished in the questionnaires. 

Questions were determined beforehand to serve as a guide to the discussion, 
during this discussion, a computer-animated demonstration of a prototype for the 
website was shown to students. This was intended to provide students with a sense of 
the possibilities for a website and provide a tangible picture where ideas for improve- 
ments and modifications could be discussed. 

Focus groups were conducted with each of the student groups. Due to schedul- 
ing conflicts, it was not possible to coordinate a focus group with the instructors that 


participated in the testing. 


DATA COLLECTION 


Four different types of data were collected: photographs, video and audio 


recordings, products of the diagram construction activity and observers’ notes. 


PHOTOGRAPHS 
Still photographs were taken to document each group’s performance as they con- 
structed diagrams of the patient case, and to record the final products resulting from 


the activity. 


VIDEO AND AUDIO RECORDINGS 

Video was recorded to document and preserve the diagram construction activity, so 
the sequence of events could later be analysed. Audio recordings were taken during 
the focus group discussions so the participants’ comments could be saved. The re- 


cordings were later transcribed onto paper to ease the process of analysis (APPENDIX C). 
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PRODUCTS OF DIAGRAM CONSTRUCTION ACTIVITY 

Each group constructed a diagram of the patient case on a white boards available in 
the room. At the conclusion of each session, photos were taken of the diagrams and 
they were transferred onto large sheets of paper. Later, the diagrams were finally tran- 
scribed into electronic format using the photos and the paper sheets as a reference. 


Careful attention was paid to the position and placement of the nodes (Ficures 23-27). 


OBSERVER’S NOTES 

Observers took notes during the diagram construction activity to record general 
impressions about the exercise. The observers were provided a review of the pro- 
cedure before the testing began with general instructions on what to watch for and 
record. The observers transcribed their notes and sent them in a text format to the 


researcher (APPENDIX D). 


Discussion 


Can medical students, with no formal visual training, construct 
meaningful diagrams of a patient case when they are provided the right 
tools, and do the constructions demonstrate tasks associated with higher- 


order thinking? 


For the purpose of this discussion, data was collected and analysed in an attempt 
to respond to the questions that were posed during the problem definition section of 
this report. 

Responses from participants in questionnaires and during focus discussion were 
examined to identify impressions and preferences about PBL tutorials, computer- 
mediated learning, and ways of approaching a patient case that may influence their 
assessment of the overall approach. 

The products of the diagram construction activity were evaluated using the 
principles of design previously discussed to determine if the students could effec- 
tively organize information visually. This evaluation, combined with participant's 
responses in the questionnaires and focus discussions, was intended to determine if 
the tools are easy to use and the task is easy to perform. 

Video recordings and observers’ notes were reviewed to determine the impact 
that the diagram construction activity had on the interactions among group mem- 
bers while they worked on the case. 

Comments written in questionnaires and voiced during the focus group discus- 


sions were analysed to determine if the participants responded to the overall activity 
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FIGURE 23: Diagram A 
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FIGURE 24; Diagram B 
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FiGurReE 25: Diagram C 
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FIGURE 26: Diagram D 
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FiGurE 27: Diagram E 
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positively or negatively and determine if there is potential to adopt the method for 
small group, problem-based learning. 

Finally, a framework for analysis that synthesized three existing frameworks for 
higher-order thinking, problem solving and decision making was developed (Figure 
28). This framework was used to analyse the diagrams constructed during the activ- 
ity, in order to provide a preliminary assessment of the potential to help students 


develop higher-order thinking skills. 


DEFINING USERS PREFERENCES 


TEACHING METHODS AND LEARNING STYLES 

To gain an understanding of the factors that may influence adoption of the proposed 
learning tool, participants were asked to respond to questions about teaching and 
learning styles, the use of specific methods on how to approach a patient case, and 
their views on computers. The responses from these questions helped to contextual- 
ize participants’ comments and determine if there is resistance to PBL tutorials or 
computer-mediated learning and why that may be the case. 

When students and instructors were asked about which teaching methods they 
consider effective, based on their personal style of learning, no single method domi- 
nated. Both groups indicated small group tutorials and clinical case scenarios were 
marginally more effective than others. While instructors seemed to prefer small 
group tutorials, students preferred clinical case scenarios. For students, lectures and 
self-directed learning exercises followed closely behind. Laboratories and interac- 
tive computer lessons, while favourable, had the most mixed responses from both 
groups. Instructors appeared to prefer laboratories and lectures less than other learn- 
ing methods. 

The fact that small group tutorials and clinical case scenarios appear to be rated 
slightly higher is not surprising. Students come to medical school to train to be doc- 
tors and the possibility to work on a life-like clinical scenario allows them to develop 
practical skills toward this end. 

During the focus groups however, students voiced a certain amount of negativ- 
ity toward problem-based learning tutorials. While most recognized the potential 
for PBL as a beneficial learning method and some had good experiences with certain 
tutors, others preferred teaching methods that exposed them more directly to clini- 
cal scenarios. Several reasons were stated, most of which focussed on issues that had 
to do with conduct of group members and group organization. Students indicated 


there was insufficient opportunity to discuss the case, as a group, because there were 
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FIGURE 28 
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certain students that dominated or there were tutors who tended to lecture. Some 
said there were tutors who were not specialists in the areas being covered and that, 
as a result of attending lectures, the students appeared to know more about the case. 
There were also concerns about the tangential topics introduced either by students 
or the tutors themselves that detracted from the discussion. Finally, there were com- 
plaints about the way some cases were written. Students said that when presented 
with a brief for the case, they could frequently skip ahead on the page, read the last 
paragraph and the solution was quite obvious. 

The information collected from students indicated they are conscious of the 
benefits that problem-based learning tutorials can offer, but possess a certain degree 
of scepticism toward PBL. Although their practical experience of PBL is somewhat 


negative, they are open to ways of improving the experience. 


COMPUTER USE 

For a proposed learning tool that will be delivered through computers over the 
Internet, it is important to understand whether students are comfortable with the 
technology. Should skills or access to technology present a barrier, a re-evaluation of 
the concept is required. Students were asked about their computer skills, habits and 
access to technology in the survey questionnaires. A summary of students responses 
follows below and graphs are provided. 

Students indicated they were all very comfortable with computer technology. 
The majority of students said they used the computer five to eight hours or more per 
week in their leisure time. The amount of time students use the computer for work 
or study in a week was slightly less. 

Every student indicated that, in a typical week, they used e-mail. Surfing the 
World Wide Web, word processing and computer-assisted learning were the next 
major set of activities conducted with computers. Only a few individuals used games, 
instant messaging, forums, conferences and blogs. 

Most students indicated they used a computer at home most frequently, fol- 
lowed by computers available to them on campus. Some had access to a computer at 
a friend’s house or in common spaces at campus residences. All students were able to 
connect to the Internet using high-bandwidth connections. 

Access to computers, the high frequency with which they are used and the high 
level of ability indicated by students suggested that technology would not present 
a barrier to acceptance of the proposed web application. It is likely that comments 
which indicate resistance to the method being proposed may come from other fac- 
tors such as, their scepticism regarding the educational benefit of PBL tutorials or 


the way the proposed learning tool may be integrated into the classroom. 
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APPROACHING PATIENT CASES 

In order to determine the level of experience solving patient cases and the potential 
of the proposed learning tool to provide a framework that helps follow a process 

to solve the patient case, it was important to determine if students and instructors 
regularly followed a process of reasoning through patient cases prior to the testing. 

As stated in the background research, there is a distinct difference between the 
way experts and novices engage in a reasoning process to approach a patient case. 
Novices tend to engage in backward reasoning which is prone to error and, once in- 
grained, it is a difficult habit to break. Experts, on the other hand, use the evidence 
they have to support possible conclusions—also called forward reasoning. 

When asked if they currently followed some kind of process to analyse and 
develop a hypothesis when they are presented with a patient case, one-third of the 
students responded that either they do not use a method at all or they do not use one 
regularly. On the other hand, all the instructors indicated they use a method for ap- 
proaching patient cases and that it was important to follow a systematic method. 

When students were asked to elaborate on why they did not use a method for 
analysis, one student indicated that time constraints and a lack of motivation dis- 
couraged them from developing a systematic approach. Another indicated they had 
not really formulated a method they could use—they tended to “take in all the in- 
formation and not organize it in a meaningful way.” A third student said it depended 
on the type of case and how much information they knew about the disease. This 
person said that the less information they knew about a disease the more systematic 
they were when approaching the case. If it were something they had seen before, they 
would likely take shortcuts. 

Typically, experts will follow a framework for approaching a clinical case that has 
been mentioned previously (p.12). Given this framework, some students indicated 


they use methods that may be considered by experts to be flawed: 


“I try to follow a structure of gathering information in a history, getting the 
information from a PE (physical exam) and labs and making a differential. 
I then refine the differential based on likelihood.” 


“T usually just read the case and wait to see what information jumps out 
at me. Most of the cases are too obvious and don’t require much thought 


process to get to the answer—not a good thing.” 


Other students, on the other hand, use methods that may be considered 
valid by experts: 
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“(1) focus on relevant history to form a differential diagnosis, prioritize the 
differential diagnosis, do physical or look at physical exam information, 


(and) run tests to confirm diagnosis.” 


“(1) usually list possible diagnoses for the major symptom/signs then use 


elimination to shorten the list of possibilities.” 


“ like to explain the presentation, build a differential from there, then use 


the physical and labs to rule things out.” 


Despite indications that students may be using both reliable and unreliable 
methods to approach a patient case, all the students indicated that a framework for 
analysis was important, valuable and necessary. A number of comments from stu- 
dents indicated that they believe a framework is important for helping deal with the 
complexity of a patient case and organize an overwhelming amount of information 


without missing important details. 


ANALYSING THE PERFORMANCE OF THE TOOLKIT 


Diagrams constructed by students and instructors were analysed using the levels of 
information explained by Bertin and Tufte and the visual characteristics proposed 
by Bertin and gestalt theory. The purpose of this analysis was to determine if the 
diagram construction was easy for students to perform without specialized training, 


and if the tools were easy to use. 


ORGANIZATION OF INFORMATION 

Groups had little difficulty organizing the information using the tools provided. The 
diagrams created by the students and instructors were analysed using the overall, 
intermediate and elementary levels of diagram reading in addition to the use of 


position, group, size and colour. 


* Overall level 

When the patient case diagrams are observed at the overall level, we can see that 
nodes are organized spatially to group common colours together with enough space 
to separate large groups of information from each other (Ficures 23-27). Diagram A 
is organized in a vertical column format, much like a list. Diagram B, C, D and E are 
organized in clusters around each of the major nodes. On all the diagrams, each of 
the clusters that contain information on patient history, physical exam, and labora- 


tory and diagnostics can clearly be distinguished. 
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* Intermediate level 

When reading the student diagrams at an intermediate level, we can see how the stu- 
dents consistently organized the information into clusters with a medium-sized node 
as a major theme or heading and small-sized nodes to indicate information contained 
within the parent node. For example, Diagram A has a medium node under patient 
history that is titled “history of presenting illness” with a series of six small nodes 
placed in close proximity. All the student groups organized information in similar 
manner, which allows the reader to quickly scan for medium-sized nodes, identify the 


title of the node and understand what may be contained within the grouping. 


- Elementary level 

Overall, the use of nodes is well suited to the elementary reading of the diagrams, but 
there were some exceptions. While most groups used a single node for a single fact, 
some groups put several facts on a node. Group A did this most frequently. As an ex- 
ample, they created a node called “associated symptoms” and listed fatigue, anorexia, 
dark urine, normal stools, mild diarrhoea, yellow eyes and olive skin. Although these 
are important facts that relate to each other as associated symptoms, in the context 
of the patient case they could be combined in different ways to mean different things. 
Splitting each of these items out into individual nodes makes it easier for the eye to 
scan the information and affords the possibility of recognizing other combinations. 

Since diagrams were completed using paper tools, stickers, erasable markers and 
a white board, it was possible for participants to improvise new ways of organizing 
information. This was the case with one of the instructor’s groups (Diacram B). After 
they went through patient history and developed a differential diagnosis, they began 
to use arrows to link up bits of information that had been placed on the diagram. 
The list of differential diagnoses they developed indicated to them that there was 
a high likelihood of exposure to a virus of some kind. They began linking signs 
and symptoms within patient history to a node they created earlier listing children 
in day-care. 

Some student groups went through a similar process of reorganizing and sort- 
ing information, however it is not visible on the diagram. Students were observed 
moving nodes to create new clusters of information or exchanging smaller nodes for 
medium-sized nodes and vice versa. 

Organizing related information in clusters to indicate relationships appears to 
assist the reading of overall and intermediate levels. However, by depending solely on 
the organization of the spatial plane, the ability to read at the elementary level begins 
to break down. Further coding of the information, provided by additional tools, will 


be needed especially at the elementary level. 
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PEREORMANCE OF ADDITIONAL TOOLS 


- Priority and responsibility indicator tools 
These tools were created with the intention of developing a way to organize and en- 
code information about a patient case that would require several hours, over a few 
days, to complete. The case that was used during the diagram construction activity 
was basic enough that it could be completed in a short period of time. In this case 45 
to 60 minutes was a sufficient amount of time to cover the basics. Most groups had 
to rush to complete the case, so certain activities were not performed. 

On the questionnaires, students indicated that they did not understand the point 
of these tools, or that instructions were too vague on how they should be used. An- 
other problem may have been that the time allocated to the task forced students to 


rush to complete the case, rather than being thorough. 


» Differential diagnosis matrix tool 

This is a tool intended for use as a guide to help student groups develop differential 
diagnoses. Only one group of students (Diacram E) and one group of instructors (Dia- 
cram B) used the matrix after they completed the patient history and continued to 
develop it as they continued through the case. 

Although the matrix was explained to the participants at the beginning of the 
testing, students indicated they forgot about it as they proceeded through the case. 
While this may be true, it may also be an indication of faulty reasoning. Often novic- 
es will rely too heavily on laboratory and diagnostic tests to prove a result. A member 


of Group A commented: 


“We forgot to do the differential before we started ordering labs and 

things. We would have ended up in the same place, but we would have had 
other stuff on the list. When we got the Hep A result, it kind of rules out 
everything else, pretty much. The only difference between how we did it this 
time and how we normally do it is that we should have had the differential 


first, but the process was still the same.” 


Considering these possibilities, the differential diagnosis matrix needs to be re- 
vised and aligned more closely with the other tools so the possibility of overlooking 


it would be reduced. 


GROUP DYNAMICS 


A desired outcome of the diagram construction activity was that it would help to 
organize group work and, as a result, enrich discussions among the members and 


reduce the potential for an individual to dominate. Video recordings and observer's 
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notes were analysed to see if the tool indeed helped to organize group activity, enrich 
discussions and facilitate collaboration. 

There was a high degree of cooperation and interaction among the participants 
(Ficure 29). At the beginning of every test, one student elected to read the case aloud 
while the others underlined important information on the sheet. The next part of 
the discussion focussed on how the construction tools should be employed to cre- 
ate a diagram. Once these aspects had been settled, students began to discuss which 
items of information should be posted where and provided a rationale. 

As the diagram construction evolved, hypotheses were explored and tested with oth- 
er members of the group; information that individuals knew was shared and elabo- 
rated upon; problems were solved and decisions were made. 

Students developed a role for themselves within the group. The roles listed here 
are defined by Bormann’s role emergence theory (Cragan et al., 2004). Role emer- 
gence theory describes a total of five roles, but only three that are applicable here: 
leader, information provider and central negative. The role of the recorder was added 
to this list. These positions were not discussed by the group members, nor strictly 


adhered to in every case. Students seemed to naturally fall into these positions. 


* Leader 

There was generally a person who took lead of the group. The leader did not force 
their opinion on the group or into following their will, but tended to be the orga- 
nizer. They kept the group on track and moving through the process. 


* Recorder 
One person took the responsibility of being the recorder. As individuals discussed, 
this person recorded information on the nodes and posted them on the white board. 


When this became an overwhelming task, others assisted. 


* Information provider 

There was also a person who served as an information resource. This person tended 
to be a natural source of information, possessing the ability to recall information 
others could not. When group members were unaware of a piece of information, this 


person would look it up in material they brought with them. 


* Central negative 

There was always one person in each group that did not like how the group was op- 
erating. Typically, they would try to change the direction of the group and their ef- 
forts were always unsuccessful. In each case, there was a moment early in the activity 
where they would make their attempt and the group would repel them by ignoring 


or trying to appease them. 
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FIGURE 29: Group interaction 


The diagram construction tookit allowed the students to work as a self-directed group to diagnose 
a patient problem. A high degree of interaction occurred between the participants of each group. 
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An example of a detractor’s influence is visible in Diagram C. After failed at- 
tempts at getting the group to consider physical exam information before finishing 
the patient history, this person created a node ordering urine analysis under labora- 
tory and diagnostics. In fact, a urine analysis was unnecessary in this case. 

After failed attempts to influence the group, the detractors usually gave up, sat 
quietly and observed the remainder of the exercise. Sometimes they would re-engage 
in the final moments of the activity. These particular students appeared to be the 
most negative about the activity in the discussions that followed. 

The diagram construction activity appeared to have the effect of neutralizing the 
students who detract the conversation. This reality shows a contrast to reports from 
students who participate in problem-based groups facilitated by tutors. Perhaps the 
activity has the benefit of providing a task to focus on, where the detractors can be 
ignored. 

The diagram construction activity provided an objective for the group to work 
toward which created a positive group dynamic. There was absolutely no discussion 
about topics outside the case, which is in contrast to the experiences expressed by 
the students in other groups. Few medical conditions that were tangential to the case 
were explored, but this particular case they had to deal with was straightforward. 
Discussions were thoughtful and informed and no individual was able to misdirect 


the group’s efforts. 


RESPONSE TO THE ACTIVITY 


Two specific questions were asked to gauge participants’ reactions about the experi- 
ence, and overall, their responses were very favourable. 

The first question asked if they thought the activity of constructing diagrams 
of a patient case, similar to the one in the study, might be a good way to prepare for 
a small group discussion with an instructor. Three instructors strongly agreed, and 
one agreed adding that it was a useful way to prepare small groups. Two of the stu- 
dents strongly agreed, ten agreed and one disagreed. 

The second question asked if they would use the technique again, and the same 
distribution among the participants occurred. Three of the four instructors strongly 
agreed and one agreed. Two of the students strongly agreed, ten agreed 
and one disagreed. 


Here are some of the written responses to these questions: 


It was a good review and a way for everyone to see the case in the same way. 


I would feel prepared to discuss this case in front of an instructor. 


I think people would actually do it and would be able to discuss them. 


This type of process is a good way to approach the problems in an organized 


and logical manner. 


The framework is effective but the tools may be somewhat time consuming 


and cumbersome to use. 


As indicated by the final comment above, students had a mixed reaction to the 
tools designed for the diagram construction activity. On the questionnaire, they were 
asked if they thought the construction tools were easy to use. Two students strongly 
agreed, four agreed, six were neutral and one student disagreed. 

The comments tended not to focus on the tools themselves, but the way in which 


they are intended to be used. This confusion is evident in their comments: 


They’re easy to use, it’s just agreeing on what goes where— 


that’s hard sometimes. 
They are straight forward, but it took time to decide how to use them. 


If you use the method frequently I think you would get used to using 


the tool. At first they seem somewhat cumbersome. 


A little bit complicated, but I think once people get used to it, it 


would work well. 


Although student comments about the diagram construction toolkit were 
mixed, overall their opinions cast the approach in a favourable light. As mentioned 
previously, the students recognized the importance of developing a framework to ap- 
proach a patient case and they considered this activity as a means of facilitating the 
development of that framework. All instructors were in favour of the approach. 

During the focus groups, students indicated that they thought this was a fea- 
sible way of supplementing problem-based learning activities and was definitely an 
improvement on what has been happening. When shown a computer-based mock- 
up of how a site could behave, they were excited by the possibilities that an online 
resource could add to the activity they had already completed. There were concerns 
about how a website, once it was developed, would be integrated into the curricu- 
lum. They did not want to have this as an activity that was added to their already 
overwhelming course load. They were also concerned about losing contact with 


the instructors. 
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INDICATIONS OF HIGHER-ORDER THINKING 


In order to determine the effectiveness of the diagram construction activity on as- 
sisting students to develop higher-order thinking skills, a long-term study would be 
required. However, a preliminary assessment was conducted using the data collected 
during this study. 

Problem solving, critical thinking and decision making are frequently described 
as three different processes. However, there are overlapping and converging activities 
that occur within each process. As such, it is difficult to identify a specific action as 
being distinct to problem solving, critical thinking or decision making. To help char- 
acterize and help distinguish these tasks, a framework for analysis was needed. 

A framework for analysis was developed by synthesizing three methods used 
to determine the achievement of tasks relating to high-order thinking. These three 
methods are: the Taxonomy of Educational Objectives established by Benjamin 
Bloom (Bloom et al., 1994); the Problem Solving Agenda Systems adapted from 
Dewey, McBurney and Hance (Cragan et al., 2004); and the Consensus of List of 
Critical Thinking Cognitive Skills outlined by the American Philosophical Associa- 
tion (Association, 1990). Through a process of finding areas of commonality be- 
tween the three models, five keywords emerged: identify, analyse, interpret, evaluate 
and implement (Ficure 28). In the following sections, these keywords are used to 
show how diagrams of a patient case may be beneficial to the development of prob- 


lem solving, critical thinking and decision making skills. 


» Identify 

The diagram constructions could help a group of students working on a patient case can 
construct diagrams in identifying pieces of data by collecting, recording and organizing 
information. 

The products of the diagram construction activity completed by the participants 
during the testing demonstrated that diagrams could be a useful means of collecting, 
recording and organizing information about a patient case. Students and instructors 
were capable of using combinations of size, colour and position in space to itemize 
elements of a patient case. The construction of a patient case diagrams made it easy 


for students to identify information contained within the clinical case. 


» Analyse 

A group of students working on a patient case could analyse the content displayed in 
diagrams in an attempt to validate information within the context of a patient’s illness, 
differentiate important positive and negative symptoms, and associate a patient’s signs 


with symptoms. 


SY 


The diagram construction activity completed by students showed that the con- 
struction of diagrams of a patient case could be a useful means of analyzing infor- 
mation related to the case. Once the information was identified and organized, stu- 
dents examined the diagram to see what may have been missing, what was important 
or unimportant and what information may have been indicating certain diseases. 

Stickers were provided in the diagramming toolkit that allowed students to rank 
information according to their needs. Some groups used these tools more effectively 


than others and some groups developed their own system of notating information. 


* Interpret 

Groups of students working on a patient case could interpret the content of diagrams to 
correlate symptoms with a disease, hypothesize about the likelihood of one disease over 
another, and articulate the data that supports the hypothesis. 

To determine the effectiveness of diagrams as tools to interpret patient infor- 
mation, the rationales group members wrote in nodes were observed and the list 
of differential diagnoses developed by students were examined. While most groups 
wrote rationales in the space provided on each of the nodes, unfortunately, due ei- 
ther to the time constraints of the test or through neglect, only two groups of the five 
developed a list of differential diagnoses as they worked. Other groups did suggest 
possible diseases in some of the nodes, without developing a differential diagnosis 
matrix as they progressed. 

Diagram B, created by a group of instructors, showed a process of linking in- 
formation in the patient history that indicates there is a possibility of exposure to a 
source of infection. They also used lines and arrows to connect information on the 
diagram that indicated jaundice. 

It appears that the process of constructing diagrams has the potential to help 
students interpret patient information. Students were able to correctly derive physi- 
cal exam and blood test information from the patient history without exploring be- 
yond the scope of the problem. All three groups solved the case successfully, discov- 
ering that the patient had Hepatitis A, but only one of the student groups successful- 


ly identified that the infection was as a result of exposure from children at day-care. 


+ Evaluate 
Groups of students working on a patient case could use a diagram to evaluate 
their conclusions by reflecting on the process, rejecting faulty claims, and verifying 
correct arguments. 
Without differential diagnoses from each group, this category is difficult to as- 


sess. When a differential is developed following the patient history, the remainder of 
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the exercise is a process of collecting information to rule diseases in or out. 

However, there are other indications that showed diagrams may serve a role in 
helping students evaluate and reflect on the process. 

Group E created a node early in the process that indicated children in day-care 
was a possible source of infection. Later in the process when they discovered Hepati- 
tis A was the likely illness, they came back and highlighted the node with an asterisk 
and a sticker. This indicates the diagram can be useful for verifying correct argu- 
ments. 

Following the completion of the activity, Group A indicated that they had made a 
mistake by not developing a differential diagnosis after completing the patient history. 
This may indicate the diagram can be useful for reflecting on the process. 

Although it has not been demonstrated that patient diagrams are useful for re- 
jecting faulty claims, their utility as a means of verifying arguments and reflecting on 


process are indicated. 


* Implement 

Groups of students working on a patient case could use a diagram to implement a plan 
of action either by prescribing a course of treatment, ordering new tests or redefining 
areas for exploration of the patient's disease. 

Within higher-order thinking, implementation is a characteristic that is unique 
to problem solving. For the purpose of this pilot study, a correct diagnosis was a con- 
firmation that the problem had been solved. One instructor group, however, showed 
signs of implementation by using the stickers that were provided in the toolkit. Once 
the group of instructors arrived at a diagnosis, they wanted people in their group to 
consider new learning issues. As a means of considering a preventative treatment, 
they wanted to know how Hepatitis A was transmitted and what are the other pos- 
sibilities for viral hepatitis. Although this activity was outside the objectives of the 
diagram construction exercise, it shows there is potential for this aspect to be incor- 
porated into the activity. 

The length of time allocated for the study, the objectives of the exercise, and the 
experience of the students may have all been contributing factors. Further investiga- 


tion is required to examine how implementation could be incorporated. 


LIMITATIONS 


The purpose of this study was to determine if medical students, without formal visual 
training, can construct meaningful diagrams of a patient case when they are provided 
the right tools, and if the constructions can demonstrate tasks associated with higher- 


order thinking. 
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The paper prototype used in this study allowed the opportunity to observe and 
analyse the dynamic of the activity to answer these questions, without the added 
burden and constraints of technology. Even though, the paper prototype served 
its purpose, more research needs to be done into the issues related to the actual 


functioning of the website, such as: 


- how the tools will function on the screen; 
- how people will interact with the tools having the computer as a medium; and 


- how group dynamics can be facilitated via the computer. 

Although the testing was valuable and informative as a means of providing use- 
ful insight into the future development of the web application, there were certain 
limitations to the study. The most significant limitations appear to be the amount 
of time allocated to the construction of the diagrams and the students’ concerns 


surrounding the tools. 


- Time restrictions of testing 

In a traditional problem-based learning setting, students have two to three hours 
per week to cover a patient case and a case may extend more than one week. During 
the test, students were asked to construct the diagrams in under an hour. In some 
instances, students rushed to finish the task. Although satisfactory data was collected 
for the identification, analysis and interpretation portion of the exercise, a more ex- 
tended test may have provided data to better examine the potential of diagrams for 


evaluation and implementation purposes. 


+ Student concerns about tools 

In survey questionnaires and focus group discussions, students indicated that the 
purpose of the tools was not always obvious. They said a lot of time was spent dur- 
ing the diagram construction activity to determine how to use the tools and, in their 
opinion, that this inefficiency reduced the effectiveness of the technique. Students 
admitted, however, that there was some learning required in order to use the tools, 
but once the confusion was overcome, it may not be as much of a factor the next 


time around. 


Design recommendations 


Design recommendations are stated in this section for later inclusion in the design 
of a computer-based version. Some of these recommendations address the improve- 


ments to be made to the diagram construction activity. Other recommendations 
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apply to the general development of the proposed web-based application. Each of 


these recommendations is based on the previous discussion. 


INTEGRATION OF THE DIFFERENTIAL DIAGNOSIS MATRIX 
The differential diagnosis matrix was underutilized during the diagram construction 
activity. It was provided to students as a separate sheet during the testing because it 
was conceived as a separate tool on the website. There were several groups that did 
not use the sheet during the diagram construction activity which indicated the stu- 
dents either forgot about the tool or there was a fault in their reasoning process. 
Students suggested incorporating the differential diagnosis as a fourth node on 
the diagram construction space. Possible diagnoses could be mapped in the same 


manner that students plot patient information. 


DIAGRAM CONSTRUCTION TOOLKIT 

Students complained during the diagram construction activity that they were unsure 
about how to use the diagram construction toolkit. The toolkit was comprised of 

six different nodes: three different colours with two different sizes. In a web-based 
version the portion of the toolkit for creating the nodes could be reduced to two 
tools—one to add a node and one to remove a node. Nested nodes could be created 
or removed by dragging and dropping a tool on the appropriate node. 

Students were concerned about the rate at which they could create nodes. They 
thought that the diagram construction activity performed with the paper prototype 
was time consuming and onerous, but they thought it might be faster to input infor- 
mation using the computer. 

One group of students suggested creating a feature on the computer that would 
allow them to highlight text presented in the patient case and transfer it to a drop 
box that would automatically create a node on the diagram. The diagram could 
then be sorted manually. Although the concept sounds feasible, the practical aspects 


would have to be designed and tested. 


NOTE TAKING 
A number of students took notes on a separate sheet of paper to remember things 
that could not be mapped on a diagram. A note taking feature should be developed 


and included, to allow students to input and share their notes. 


SEARCH, SORT AND SELECT 

The ability to search, sort and select large or small groups of related information 
can aid the process of analysing, interpreting and evaluating the patient case. Tools 
to tag information with properties such as priority, responsibility, urgency and rel- 


evance should be developed and incorporated in the computer prototype for testing. 
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Although these tools were underutilized in the testing of the paper prototype, the 
capability of the computer to perform these operations is much better and may be 


more beneficial. 


INSTANT REPLAY 

A feature that allows a viewer to play through the chronological sequence of the dia- 
gram construction may be of some value in analysing and evaluating the products 
of the group activity. The order of the placement of the nodes was helpful to the 
researcher in the examination and analysis of each groups’ work completed during 
the testing and it may also have a benefit to instructors of PBL tutorials. The feature 


could allow them to play back each diagram at any rate they wished. 


DIAGRAM COMPARISON 

Students indicated that they would like to know what occurred in other small group 
PBL tutorials. A feature could be constructed that allows students and instructors to 
do a side-by-side comparison of all the diagrams constructed by all the groups. This 
would allow students to select and explore diagrams that were different than their 
own to discover why and would also allow instructors to compare the work of one 


group against another. 


COMMUNICATION FUNCTION 

Most students indicated they preferred to create the diagrams together in front of the 
same computer. While a communication function would not be necessary for this 
purpose, they also wanted to be able to contact their tutor to ask questions. Discus- 
sion forum software should be researched and integrated into the whole package. 
Common questions could be shared across several groups and tutors could use this 


as a forum to make announcements to the group. 


ADMINISTRATIVE FUNCTIONS 
In the paper prototype testing, much of the administrative functions were performed 
by the researcher. To coordinate this activity on the computer, a mechanism will have 
to be designed and developed that will automatically direct students to their assigned 
group when they enter the website. Forms for instructors to input case content that 
could consist of text, numerical data, image, audio, video or animation will also have 
to be developed. 

Students liked how information was revealed to them in pieces as the case pro- 
gressed. A system will also have to be developed that allows students to request and 


receive information, such as physical exam and laboratory results. 
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SUMMARY OF RECOMMENDATIONS 


To translate the diagram construction activity performed on paper to a computer- 
based version for further testing, the following additions and modifications 


are recommended: 


- A fourth space on the diagram for a node to develop differential diagnoses. 


- A feature that allows a user on a computer to select and drag text from the case 


description into a drop box that automatically generates nodes on the diagram. 


- A note taking feature that allows students to input and share 


general comments. 


- A function that facilitates the ability to search, sort and select groups of 
related information to aid the analysis, interpretation and evaluation of 


patient information. 


- An instant replay function that allows a user to play back the chronological 
sequence of the diagram construction activity may help analyse and evaluate 


student performance. 


- A feature that shows side-by-side comparison of the diagrams created by 


students may help analyse and evaluate student performance. 


- A communication function that will allow students to contact each other 


and the tutor. 


- An administrative function that will automatically direct students to their 


assigned group when the enter the site. 


Implications for future research 


The results of this study have yielded an interesting set of opportunities for future 


research. These are outlined as follows: 


USABILITY TESTING 
The next logical step in the development of this project would be to create a func- 
tioning prototype of the website. The ability to create an interface that is both re- 


sponsive to the needs of the user and intuitive is important for the continuing suc- 


cess of the project. Usability testing will be required to gauge how users will respond 


to the look and the navigational components and if they serve the purpose for which 


they are intended. 
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CLINICAL REASONING MODEL 

The organization of nodes into patient history, physical exam and laboratory and di- 
agnostics may be beneficial to help students develop clinical reasoning skills through 
problem solving, critical thinking and decision making. Physicians have made sug- 
gestions for other organization of nodes that could enhance other types of learn- 
ing around a patient case. Explorations of the pathophysiological mechanisms of a 
patient case or an approach to evidence-based medicine are two such examples cited 
by physicians. Further investigation is required to see how the configuration of the 


primary nodes influences the result. 


CLARIFY TASK OBJECTIVES 

In order to assess, in greater detail, whether diagrams can perform the evaluation 
and implementation characteristics identified in the problem solving, critical think- 
ing and decision making framework, clarification of the task objectives presented to 
students may be required. More testing that provides additional instructions requir- 
ing students to, not only construct a diagram, but also develop a list of differential 


diagnosis and a treatment plan may be necessary. 


CREATION OF PATIENT CASES 

The creation of this website is only one part of the equation. Without patient cases 
on a diverse range of topics within medicine, this website cannot function as a tool 
to facilitate problem-based learning tutorials. Physician/educators who lack experi- 
ence writing patient cases could be provided with a guide to aid the creation of cases. 


An intuitive interface to input the cases into the system will also be required. 


DIAGRAMS AS AN EVALUATION TOOL 
One of the intentions of this website was for instructors to be able to monitor and 
provide feedback to students constructing diagrams of patient cases. Due to schedul- 
ing and technological factors, it was impossible to collect instructors’ impressions. 

It appears that this technique may have potential as a means of evaluating prob- 
lem solving, critical thinking and decision making skills of groups in a problem- 
based learning context. Research should be undertaken to confirm this hypothesis 


and develop guidelines for how instructors may evaluate diagrams. 
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Conclusion 


Physicians must be skilled at evaluating a patient's illness, identifying symptoms 
through reasoning and research, and making a correct diagnosis of the medical 
problem in an efficient manner. Expert skills at problem solving, critical thinking 
and decision making are required of a physician to carry out these responsibilities. 
Problem-based learning tutorials are intended to help medical students develop 
these skills while working in a small group on a patient case. However, medical stu- 
dents require feedback on their problem solving, critical thinking and decision mak- 
ing from tutors in order to improve. 

This project combined theories that show the potential of diagrams to facilitate 
higher-order thinking with the capability of group coordination shown by com- 
puter-based group support systems to produce an e-learning tool for problem-based 
learning tutorials. This tool could help instructors provide feedback to groups on 
their reasoning and problem solving process while providing logistical support to 
small groups studying a patient case. 

A study was conducted to assess the performance of a paper prototype of the 
proposed tool, and students’ and instructors’ responses to the activity. The results 
of the test showed that students can construct meaningful diagrams when given the 
right tools. Students and instructors also showed acceptance of the approach and in- 
dicated that they were motivated to use it again. 

Comments provided by students were also helpful in determining recommenda- 
tions for the design of a computer-based version of the diagram construction activ- 
ity and helped define possibilities for future research. This project initiates the for- 
mative stages of the process. More work needs to be done to further validate findings 


of this study—both on the technical and pedagogical fronts. 
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Application to Arts, Science and 
Law Research Ethics Board 


« Submission documents 


- Research approval documents 


APPENDIX A 


UNIVERSITY OF ALBERTA 


Arts, Science & Law Research Ethics Board (ASL REB) 
Certificate of REB Approval for Fully-Detailed Research Proposal 


Applicant: Marc Brisbourne 
Supervisor (if applicable): Prof Jorge Frascara 
Department / Faculty: Faculty of Arts, Department of Art & Design 


Project Title: Constructing Diagrams in Small Groups as a Method to Help Medical Students Learn 
Clinical Problem-Solving and Decision-Making Skills 


Grant / Contract Agency (and number): 
ASL REB Member (and file number if applicable): LKP (Chair) 
Application number: ASL-REB # 801 


Approval Expiry Date: January 10, 2006 


CERTIFICATION of ASL REB APPROVAL 


I have reviewed your application for research ethics review and conclude that your proposed research 
meets the University of Alberta standards for research involving human participants (GFC Policy Section 
66). On behalf of the Arts, Science & Law Research Ethics Board (ASL REB), I am providing expedited 
research ethics approval for your proposed project. 


Expedited research ethics approval allows you to begin your research with human participants, but is 
conditional on the full ASL REB approving my decision at its next meeting (17 January 2005). If the 
full ASL REB reaches a different decision, requests additional information, or imposes additional 
research ethics requirements on your study, I will contact you immediately. 


If the full ASL REB reverses my decision, and if your research is grant- or contract-funded, the Research 
Services Office (RSO) will also be informed immediately. The RSO will then withhold further funding 
for that portion of your research involving human participants until it has been informed by the ASL REB 
that research ethics approval for your project has been granted. 


This research ethics approval is valid for one year. To request a renewal after (January 10,2006), please 
contact me and explain the circumstances, making reference to the research ethics review number 
assigned to this project (see above). Also, if there are significant changes to the project that need to be 
reviewed, or if any adverse effects to human participants are encountered in your research, please contact 
me immediately. g ‘ = 


ASL REB member (name & signature | 
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UNIVERSITY OF ALBERTA 
FACULTY OF ARTS, SCIENCE & LAW 
RESEARCH ETHICS BOARD 


APPLICATION TO CONDUCT RESEARCH 
INVOLVING HUMAN PARTICIPANTS 


Principal Investigator(s): 
Name: Marc Brisbourne 
Department/Faculty: Art & Design, Faculty of Arts 
Campus Address: 3-98 Fine Arts Building 
Campus Phone number: 492-1309 


E-mail address: marc.brisbourne@ualberta.ca 

(If student) 
Name / Department of Faculty Supervisor: Jorge Frascara, Art & Design 
Supervisor's E-mail address: frascara@ualberta.ca 


Supervisor's Campus Phone number: 492-5092 


Project Title: Constructing diagrams in small groups as a method to help medical 


students learn clinical problem-solving and decision-making skills 


Funding Source(s): N/A 


DATA IS TO BE COLLECTED OUTSIDE THE FACULTY OF ARTS 


TABLE OF CONTENTS 


Summary of Project/Research Design. .......-..+++.5+5- 1 
Assessment of Risk to Human Participants ...........+.: 2 
Description of Procedures to Reduce Risk ............... 3 
ECHO Ope SIMPBIOLOM codon dono noc vdoeoBoanesbor Appendix A 
ReEeHUMtMeNt eltehaian tha ttt irr ra ae Appendix B 
Study Explanation ies. cim eerie ys ys ier Appendix C 
Task Synopsis sheets... ree ei Appendix D 
Participant Consent Form............+....-+. Appendix E 
iDEN MEANS concn ancobocods ovo cdmoosOnans< Appendix F 
AS RACINE OUISSUOTMIAEWR®, ooo oc no nnonecnoeoos Appendix G 
Instructor Focus Group Questions............ Appendix H 
Student Ouestionnalvel wetter tert t terri Appendix | 


Student Focus Group Questions .............. Appendix J 


SUMMARY OF PROJECT / RESEARCH DESIGN 


Please attach a more detailed proposal (i.e., 1-2 pages), including a description of the popula- 
tion from which research participants will be drawn (e.g., university students, nursing home 
residents) and a discussion of how research participants will be solicited. Also attach copies 


of research instruments (e.g., questionnaires, interview guides). 


The development of expert critical thinking, problem-solving and decision- 
making skills to be able to evaluate and manage a patient in an efficient, effective 
and humane manner is an important skill for medical students. Studies have shown 
that novices tend to employ a form of backward (hypothetico-inductive) reasoning 
where they establish a hypothesis and try to prove it to be true. While medical stu- 
dents can produce elaborate explanations, their assessment is prone to a high rate 
of error. Expert physicians, on the other hand, have a stronger tendency to examine 
all the facts before offering an opinion. The findings of these studies suggest that 
junior medical students may not always be receiving sustained quality feedback of 
their critical thinking and problem-solving skills, nor can they develop those skills 
adequately without proper guidance. 

The purpose of this project is to initiate the development of an instructional web 
site that would allow small groups of medical students studying clinical cases to 
construct diagrams as a means of developing and improving critical thinking, de- 
cision-making and problem-solving skills. Once created, these diagrams would 
allow students to reflect and correct their own work, while providing instructors 
the opportunity to assess student performance and provide comments. Through a 
mechanimsm that provides an opportunity for self-reflection and feedback from in- 
structors, students may be better able to develop skills to behave more like an expert 
when they treat a patient. 

This study is intended to help determine the feasibility of the concept and answer 
questions about the design and development of the web site by observing groups of 
students working in small groups constructing diagrams of a clinical case on paper. 
The findings of this study will help determine the potential for success of the web 
site concept and assist in identifying adjustments that should be made to the overall 
design to improve the potential for successful implementation. 

Four expert physician/educators and 16 second-year medical students from the 
Faculty of Medicine and Dentistry will be asked to participate in the study. Each 
person will be asked to take part in an exercise that involves creating a diagram of a 
clinical case, they will be asked to fill out a questionnaire and participate in a focus 


group to collect feedback. The combination of diagram construction, questionnaire 


and focus group will take approximately two hours. The experts will work indepen- 
dently and the students will work in groups of four. 

Participants will be recruited through contacts in the Faculty of Medicine, the of- 
fice of Undergraduate Medical Education in the faculty and the University of Alberta 
Medical Student’s Association. Physician/educators will be recruited from within 
the Faculty of Medicine and Dentistry through recommendations from their peers. 
Medical students will be recruited by asking a second-year medical student represen- 
tative to address the class and through a mass e-mail to the same group of students. 

The study will be conducted in a meeting room on the fifth floor of the Medical 
Sciences Building provided by the Division of Anatomy in the Faculty of Medicine 
and Dentistry. Participants will be provided lunch as an incentive to participate in 


the study. 


ASSESSMENT OF RISK TO HUMAN PARTICIPANTS 
[Attach additional page(s) if necessary} 


The risk of human participants exposed to harm during this study is minimal. 


Group Activity and Interaction 
Volunteers will be asked to participate in a simulated small group task with fellow 
students. In some situations conflict can arise, however, the potential for conflict is no 


greater than that of any classroom exercise conducted in a course at the University. 


Patient Case Content 

Participants will be asked to engage in a reasoning and problem-solving task to 
reach a differential diagnosis about a simulated patient case. The case may deal with 
information that a participant may find disturbing, however, the risk is minimal. The 
medical case selected for this exercise was written by and is currently in use at the 
Northeastern Ohio Universities College of Medicine. The type of information con- 
tained in the case is typical of the level of information a medical student may experi- 


ence in routine activities in medical school. 


Data Collection and Privacy 

Video, audio and still images of the group activities will be captured during the 
study for later analysis. The researcher will ensure the confidentiality of the partici- 
pants. Images and quotations from the participants may be used in a public exhibi- 
tion that is a mandatory component of the Master of Design thesis. To protect their 


identity, faces and names will not be associated with their picture or their words. 


No 


DESCRIPTION OF PROCEDURES TO BE UNDERTAKEN 
TO REDUCE RISK TO HUMAN SUBJECTS. 


[Please attach copies of consent forms and other similar documents. ] 


Participation in the task exercise, questionnaire and focus group is voluntary. An 
invitation to participate will be delivered to students, in class, by a representative 
of the second-year medical student class. The invitation will outline the project, de- 
scribe the participant's role, the approximate time commitment and encourage their 
participation (see Appendix B). The verbal invitation will be followed up by an e- 
mail invitation to the entire class. 

Volunteers will be assigned a date to participate. A member of the research team 
will explain the risks and benefits of participating in the evaluation, remind the 
students that their participation is voluntary and inform them that they may leave 
whenever they would like (see Appendix C). 

The risks and benefits explained by the researcher before the evaluation begins 
will address the items mentioned in the assessment of risk section above. The re- 
searcher will explain: 

(a) The evaluation requires participation in an exercise that requires small group 
interaction. The exercise should be treated as a classroom exercise and, as such, inap- 
propriate behavior will not be tolerated. If a conflict arises, the person observing the 
exercise will stop the activity; 

(b) The exercise will deal with a patient case that may contain sensitive issues for 
some individuals. Although the content is no different than what they were exposed 
to in the Gastrointestinal block during their second year of instruction, if students 
find the material disturbing, they may leave the exercise at any time; 

(c) Audio, video, and still images will be recorded for later analysis and may be 
used in a public exhibition that is a required component of the thesis. To protect an 
individual's privacy, steps will be taken to ensure their face or name will not be as- 
sociated with their picture or words. Audio recordings will be taken to preserve the 
discussions. Transcripts will be made of the discussions, but names will be removed. 
Video recordings will only be taken of the diagrams so the flow of the construction 
can be preserved for later analysis. Still photographs will be taken of the activity 
for display in a public exhibition, but the participant’s faces will not be revealed. 
Participants will be allowed the option of not having their image shown in the exhi- 
bition. 

The volunteers will be asked to fill out a consent form that indicates they under- 


stand the risks and they consent to participate (see Appendix E). 
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I have read the UNIVERSITY OF ALBERTA STANDARDS FOR THE 
PROTECTION OF HUMAN RESEARCH PARTICIPANTS [GFC Policy Manual, 


Section 66] and agree to abide by these standards in conducting my research. 
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January 13, 2005 


Dr Lynn Penrod, Chair 

Arts, Science, Law Research Ethics Board 
Department of Modern Languages & Cultural Studies 
200 Arts Building 


Dear Dr Penrod: 


This is simply a letter in support of Mr Brisbourne’s Masters-level thesis project. 


| have been consulting with Marc over issues related to the development of this, and | believe 
this will be valuable research to assess how students reason and formulate hypotheses when 
working on clinical problems. | have read Marc’s research method, and | believe this will be a 
valuable adjunct to his work in educational technology design, as well as to us in the Faculty of 
Medicine and Dentistry. 


In any case, | see no significant problems with this from an ethical standpoint as far as obtaining 
information from our students and faculty, and | hope your Research Ethics Board can expedite 
his application. If | can provide any further information, please do not hesitate to contact me. 


| Education 
oratory Medicine & Pathology 


Office of Admissions & Undergraduate Medical Education 
Faculty of Medicine and Dentistry 
2-45 Medical Sciences Building * University of Alberta - Edmonton: Alberta « Canada : T6G 2H7 
Telephone: (780) 492-6350 Fax: (780) 492-9531 
www.med.ualberta.ca 
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Appendix B 
LTD ry rt Division of Anatomy 
YW DILIN £ ay Faculty of Medicine and Dentistry 


5-01 Medical Sciences Building Tel: 780.492.2628 
Edmonton, Alberta, Canada T6G 2H7 Fax: 780.492.0462 


January XX, 2005 
Dear Student, 


Studies have shown that a large proportion of medical students engage in a clinical reasoning and problem solv- 
ing process that differs significantly from that of an experienced clinician. Without proper support, the develop- 
ment of expert clinical reasoning and problem solving skills can be impeded, which has serious implications for 
physicians as they enter practice. We are engaged in a research project to conceive and develop a web site that 
would serve to facilitate the development of expert clinical reasoning and problem solving skills of medical stu- 
dents covering simulated patient cases. 

We invite you to participate in an evaluation of the concept, approach and feasibility of a web site to supplement 
small group learning in the faculty. We are looking for 16 student volunteers to provide us input and opinions. 
Your feedback is very important to us, so we can understand if the approach is feasible and determine how we can 
improve the concept. Through this research project, we hope to ensure the educational material that may be devel- 
oped in the future is suitable to your needs and is of the best quality available. 

Your contribution to this research will be to work on a patient case involving material from the Gastrointestinal 
block in a small group, fill out a brief questionnaire and participate ina focus group. Your commitment will take 
approximately two hours on one of the days indicated below: 

Day 01: January xx, 2005 xx:00 pm to xx:00 pm 

Day 02: January xx, 2005 xx:00 pm to xx:00 pm 


This research study is being conducted in cooperation with the following investigators. Please feel free to contact 


one if you have any questions or concerns about the study. 


Marc Brisbourne Graduate Student, Art and Design 492-1309 
Jorge Frascara Professor, Art and Design 492-5092 
David Begg Professor, Anatomy 492-7179 
David Rayner Associate Professor, Laboratory Medicine & Pathology 407-6012 


If you are interested in participating, please send your contact information and preferred date to Marc 
Brisbourne by one of the following means: 

E-mail: mbrisbou@med.ualberta.ca Phone: (780) 492-1309 

Address: Division of Anatomy, 5-05D Medical Sciences Building 


Thank you for your interest in this study, 


Sincerely, , 
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Testing Materials 


* Study explanation 

* Task exercise 

* Participant consent form 

* Patient case 

- Instructor questionnaire 

- Instructor focus group protocol & questions 
* Student questionnaire 

* Student focus group protocol & questions 
* Diagram construction nodes 

* Physical exam & test result samples 

» Differential diagnosis matrix 


* Responsibility, priority & urgency stickers 


APPENDIX B 


STUDY EXPLANATION 


Thank you for taking the time to participate in this study. My name is Marc 
Brisbourne. I am a graduate student in the Department of Art & Design doing a 
Masters in Visual Communication Design. For my thesis topic, I have decided to 
approach the design and development of a web site for small group learning that is 
based on the cooperative construction of diagrams to facilitate the development of 
critical thinking, decision-making and problem-solving. I am undertaking this with 
support from faculty members from Medicine and Dentistry. Four of your instructors 
participated in a similar study to this one just last week. You will have an opportu- 
nity to look at their results and compare them to your own later on. 

This study should take approximately two hours today and will be broken into 
three phases: a task exercise, a brief questionnaire for you to complete, and a focus 


group discussion. Lunch will be served at the start of the focus group. 


Phase I; Task Exercise (45 minutes) 

The task exercise is intended to simulate a small group, problem-based learning 
situation covering a patient case that could occur in the Gastroenterology block. You 
will be assigned to a group with three others and you will be provided a written pa- 
tient presentation. You have been to lectures this past September that have provided 
you with enough of a knowledge base to approach the case. In this scenario, we want 
you to pretend that you have been asked by your instructors to prepare for the first 
meeting of your small group about the case ahead of time by: 

¢ Identifying important aspects of the patient presentation; 

* Mapping information that is known about the patient and identifying infor- 
mation that may be required; 

° Describing the importance of each of the pieces of information to the case 
(if it is appropriate to do so); 

¢ Indicating the relevance, importance and responsibility for each piece 
of information; 

° Offering a list of differential diagnoses for the case as it is presented. 

Your task is to create a diagram as a group that will accommodate each of the 
objectives just mentioned. Although this exercise is intended for eventual construc- 
tion on the computer, you have been provided with paper toolkits that simulate the 
interactions that could occur. To begin the diagram, your group will be provided with 
three major nodes: Patient History, Physical Exam and Laboratory and Diagnostics. 
Your toolkit contains various color coded nodes. You must decide how these will 
be used and what they represent. On the node, there is space to: write a title; assign 


responsibility for the content of the node to a member of the group; indicate the pri- 


ority of the information; and indicate the relevance of the information contained in 
the node to the case at hand. Inside the node there is a piece of paper that provides 
space for a title, the facts that relate to the topic and rationale for how the facts relate 
to the case. 

You will have 45 minutes to map out as much of the case as possible so you can un- 
derstand the patient’s condition, the diseases that may be underlying the symptoms 
and develop a set of differential diagnoses for the condition on the matrix provided. 

An assistant will be assigned to your group to observe and record your process. 
The assistant possesses some additional information that may be beneficial to solving 
the case. If, during the process, a node is created for the information which you do 


not possess, the assistant will provide it to your group. 


Phase II; Questionnaire (30 minutes) 

After completing the task exercise, you will be asked to fill out a brief question- 
naire. This questionnaire asks some specific questions about the instructional ap- 
proach and your experience completing the task. The questionnaire will allow you to 


rate your responses and provides space for more detailed feedback. 


Phase III: Focus Group Discussion (45 minutes) 

The focus group discussion is intended to get both groups together to compare 
results with the other groups and their instructors and talk about their experiences 
completing the case. Although there are some specific questions we would like to 
discuss, the format is flexible depending on issues that may arise during the task 


exercise. 


Risks and Benefits of Participation 

There are minor risks associated with participating in this study that I am obli- 
gated to inform you about. 

The task exercise is a group activity that involves interaction with your peers, and 
sometimes conflict can arise. You are reminded that participation in this study is a 
University activity and you are bound by the code of student conduct enforced by the 
Faculty of Medicine and Dentistry. If a situation arises that cannot be solved within 
the group, the assistant will stop the exercise. 

The task exercise covers a simulated patient case and individuals may find some 
of the material disturbing. The material, however, does not differ significantly from 
what you may have covered during the Gastrointestinal block. If you are uncom- 
fortable with any of the material, you may excuse yourself from the exercise at any 


time. 
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Still images, video and audio recordings will be taken during the study. The data 
collected using these devices will be used for analysis. Images and quotations from 
the audio transcripts may be used in a public exhibition as part of the thesis exam. 
You will not be identified by name. If you do not wish to have your picture used in 
the exhibition, please indicate your preference on the consent form that has been 
provided to you. 

By participating in this study, you will be influencing how your fellow medical 
students may be educated in the future and how new media applications created in 
the faculty are developed. The faculty is interested in providing students with the 
best and most advanced educational opportunities possible so they are equipped 
with the proper skills and training to enter professional practice. Your participation 


in this study is greatly appreciated by members of the faculty and myself. 


The Private and Voluntary Nature of Participation 

I am to remind you that your participation in this study is entirely voluntary. If 
you wish to leave, you are entitled to do so at any time without need for justification. 
Your privacy will be protected from individuals outside this study. Your name will 
not be associated with any response. The performance of you or your group during 
this study will have no bearing on your academic performance. 

If you agree to these conditions, please fill out the consent form. We will then di- 


vide into groups and begin the activity. 
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SMALL GROUP, PROBLEM-BASED LEARNING TASK EXERCISE 


Purpose: To prepare, as a group, for a problem-based learning tutorial by mapping out the 
information for a patient case in a manner that reflects the process you might use to approach 


the evaluation, assessment and management of a patient that has come to visit you in your 


office. 
Duration: 45 minutes 


Questions to consider during the construction of the diagram: 
What is/are the patient’s problem(s)? 
What are the possible causes of his condition? 
What additional information may help you distinguish these possibilities? 
What aspects of the patient’s history are important? 
What aspects of the physical exam are important? 
Would you order any laboratory tests? If so, which ones and what do you 


hope to learn from the results? 


While covering the patient case: 

e Identify important aspects of the patient presentation; 

¢ Map information that is known about the patient and identify 
information that may be required; 

° Describe the importance of each of the pieces of information to 
the case (if it is appropriate to do so); 

° Indicate the priority, responsibility and urgency of information; 


e Offer a list of differential diagnoses for the case as it is presented; 


Materials: 

¢ Three large nodes (history, physical exam, and laboratory and diagnostics) 
e Aset of medium nodes (blue, green and purple) 

e A set of small nodes (blue, green and purple) 

e <A differential diagnosis matrix 

e Stickers for to indicate priority, responsibility, and urgency 

e Markers 

¢ Highlighters 

e Masking tape 


PARTICIPANT CONSENT FORM 


Part 1: Researcher Information 


Name of Principal Investigator: Mare Brisbourne 
Affiliation: Graduate Student, Department of Art & Design, Faculty of Arts 
Contact Information: 5-05D MSB, (780) 492-1309, marc. brisbourne@ualberta.ca 


Name of Thesis Supervisor: Jorge Frascara 
Affiliation: Professor, Department of Art & Design, Faculty of Arts 
Contact Information: 3-98 FAB, (780) 492-5092, frascara@ualberta.ca 


Part 2: Consent of Participant 


Yes 


Do you understand that you have been asked to be in a research study? 


le 
° 


Have you received a copy of the study information sheet and has it 
been explained to you? 


Do you understand the benefits and risks involved in taking part in this 
research study? 


Have you had an opportunity to ask questions and discuss the study? 


Do you understand that you are free to refuse to participate or withdraw 
from the study at any time? 


Do you understand that still images, video, audio recordings will be 
taken during the study? 


Do you permit the researcher to use your image or comments in 
a public exhibition? 


Has the issue of confidentiality been explained to you? Do you understand 
that your name will not be published in any material related to this study? 


Part 3: Signatures 


This study was explained to me by: Mare Brisbourne 
Date: 


I agree to take part in this study. 


Signature of Research Participant: 


Printed Name: 


PATIENT: GEORGE CROOKER 


George Crooker, a 33 year-old Caucasian male, comes to your office because he has 
a bellyache, nausea, and has not felt well for the past three days. He has noticed in 
the mirror that the whites of his eyes are unusually yellow and that his skin has an 
olive coloration. 

Mr. Crooker has always been employed as a medical technician. He is married 
and the father of two children, aged 4 and 2, who are in the day-care at the Happy 
Elephant Play Academy. No one else in the family is ill. Mr. Crooker has been very 
fatigued over the past three days, has had no appetite and is experiencing mild diar- 
rhea. He said that he has not been traveling and has not taken any drugs and has only 
consumed a couple of beers in the past few weeks. 

He became irritated when questioned about his sex life. He emphatically stated 
that he was a married man. 

There is no family history of jaundice, but his grandfather once had gallstones. Mr. 
Crooker’s urine has been brownish recently, but he has not noticed any changes in 
his stools. There have been no referred pains. He has no history of surgery or blood 


transfusions. 


Additional information: 


Height: 5’9” Heart rate: 90 beats per minute 
Weight: 160 lbs. Respiration rate: 20 per minute 
Blood pressure: 120/66 Temperature: 39°C 


Questions to consider during the construction of the diagram: 


What is/are the patient’s problem(s)? 
What are the possible causes of his condition? 


What additional information may help you distinguish these possibilities? 


What aspects of the patient’s history are important? 


What aspects of the physical exam are important? 
Would you order any tests? If so, which ones and what do you hope to 


learn from the results? 
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INSTRUCTOR QUESTIONNAIRE 


This questionnaire consists of 16 questions on four pages. When asked to respond to a statement, please in- 
dicate whether you strongly agree, agree, neither agree nor disagree, disagree or strongly disagree. Please 


explain your answer whenever appropriate. 
Preferred Methods of Instruction 


1) Anumber of learning methods are employed in the teaching blocks from first and second-year 
studies in medicine. According to my preferred style of teaching, I find each of the following learning 


methods very effective. 


Strongly Agree Neither Disagree Strongly 

agree A nor D disagree 
Lecture O O O O O 
Laboratory O 0 0 i) 
Small group tutorials O O ja O O 
Self-directed learning exercises O i) O ie O 
Interactive computer lessons O O O O O 
Clinical case scenarios O O O O a 
Other (specify: Yee el O a) O O 


2) Rank your preferences for the teaching methods employed by instructors in each of the teaching 


blocks from first- and second-year studies (1 indicates most preferred and 10 indicates least 


preferred). 

____ Introductory block ____ Gastroenterology and Nutrition 
____ Infection, Immunity and Inflammation ____ Reproductive Medicine & Urology 
_____ Endocrinology ____ Musculoskeletal System 

____ Cardiovascular, Pulmonary and Renal ____ Neurosciences 

____ Patient-Centred Care ___ Oncology 


Briefly describe why you believe the teaching methods of the top ranked block work well. 


3) The level of difficulty of the content of the case covered in this research study is on par with material 
covered in the first and second year of medical training at the University of Alberta. 


Strongly agree O Agree 0 Neither A nor D 0 Disagree 0 Strongly disagree 0 


Approach to Patient Cases 


4) 


5) 


6) 


7) 


When I tutor students covering a patient case in class, I encourage them to develop a method or 
framework to analyse and develop a hypothesis about a patient’s condition. 


Strongly agree O Agree 0 Neither A nor DO Disagree O Strongly disagree 0 


Explain: 


I think it is important that students use a method or framework to analyze and solve a patient case. 


Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree 0 


Explain: 


I think the diagram construction activity in this research study would be helpful in allowing students 
to analyze and understand a patient problem in order to make a differential diagnosis. 


Strongly agree 0 Agree 0 Neither Anor DO Disagree 0 Strongly disagree 0 


Explain: 


I think the activity would help a group of students to understand aspects of a case that they may not 
have otherwise realized. 


Strongly agree 0 Agree 0 Neither A nor DO Disagree 0 Strongly disagree D 


Explain: 


8) I think the activity of constructing diagrams of a patient case is a good way for students to prepare 


for a small group discussion with an instructor. 


Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree O 


Explain: 


9) Ithink the diagram construction tools provided for the task exercise were easy to use. 


Strongly agree 0 Agree OU Neither A nor DO Disagree O Strongly disagree O 


Explain: 


10) I would encourage students to use a similar process to solve a patient case as a group in the future. 


Strongly agree O Agree O Neither Anor DO Disagree O Strongly disagree 0 


Explain: 


Personal Use of Technology 


11) Iam very comfortable using computer technology (web browsing, e-mail, word processing, etc.). 


Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree 0 


12) During my leisure time in a typical week, I estimate I use the computer: 


Less than 1 hour O 1 to 4 hours O 5 to 8 hours 0 9 to 12 hours O 13 or more O 


13) During my work/study time ina typical week, I estimate I use the computer: 


Less than 1 hour O 1 to 4 hours O 5 to 8 hours O 9 to 12 hours O 13 or more O 
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14) Ina typical week, I frequently use a computer for the following: 


Strongly Agree Neither Disagree Strongly 

agree A nor D disagree 
E-mail fi} O O O O 
Instant messaging O O O O O 
Forums /Conferencing/Blogs Oo [a a Oo Oo 
World Wide Web surfing O Oo O O O 
Computer-assisted learning O O O O O 
Word processing O O i) ial Oo 
Games O O O O Oo 
Other (specify: ) (a O ja) O O 


15) Rank the location of the computers you use to access web sites and e-mail with five being the most 
frequent and one being the least frequent. Only number the spaces that apply to your habits for using 


the Internet. 


Home Computer on the wards 
Campus computer lab Office 
Other (please specify: ) 


16) How do you most often connect to the Internet? 


Dial-up modem Oo High-speed Internet Oo Wireless 0 None 0 


INSTRUCTOR FOCUS GROUP PROTOCOL 


Context 

As I stated earlier, the purpose of this study is to collect student and instructor feedback on the de- 
sign of a method to supplement small group, problem-based learning groups in the Faculty of Medicine. 
Ultimately, our desire is to create a web-based application that would allow students to construct these 
diagrams as a group. I’ve brought some mock-ups of what this application might look like to show you and 
I have some questions to ask you about the exercise we just did and about this concept. We'll get to that in 


just a few moments. First I wanted to show you what this program might look like: 


Figure 1: Login screen 
This screen would allow you to sign-in with a unique user id and be automatically assigned to a 


group, depending on the teaching block you happen to pewn: 


Figure 2: Course and case selection screen 
This screen would allow you to select the teaching block in which you happen to be studying and 
select the case that you’ve been assigned to work on. You will also be able to review your diagrams and 


the diagrams of other groups from previous exercises. 


Figure 3: Case presentation 
This screen shows the patient presentation with a text description and a picture. Blood tests, X-Rays, 
EKG readouts or any other diagnostic results could be included. Video and audio components could 


also be incorporated. 


Figure 4: Diagram construction interface 
This screen shows some suggestions for tools necessary to create a diagram on the screen ina similar 
fashion to what you have just completed with paper. You can create a node, label it, fill out important 
information, write a rationale and tag the node with data that could be helpful for you to organize 


yourself. 


Figure 5: Search and sort 


This screen shows how you could use the properties to sort the diagram. 


I wanted to ask you some questions about your experience creating the diagram and 


about some of the things I’ve just shown you for a web site. 


Questions 


1. How did you enjoy the activity that you completed earlier? Why or why not? 
2. Are small group or problem-based learning sessions conducted in the block in which you teach? 
Why or why not? 
Do you encourage students to use a method of approaching clinical cases? What is it? 
Do you think a method like the one used today was a good way to prepare for small group 
discussions? Why or why not? 
5. After completing this task exercise today, do you think it might get students to think differently 
about the way they approach a patient case? Why or why not? 
6. Do you think a web-based resource to prepare for small group discussions, something like 
the one I showed you, would be beneficial? How? Why? 
Would you utilize a web resource such as the one I showed you? Why or why not? 
Would you prefer to see students create the diagram sitting together in front of a computer or 
would you prefer them to be able to do it as a group from remote computers? Why or why not? 
9. The web site would contain many of the features of the paper tools that you used today in 
the activity — labelling, rationale, description, relevance, priority, etc. Did you find those 
features useful? Why or why not? 


10. What additional features or tools do you think would be useful in such a web site? 


That is all the questions I have for today. Does anyone have anything they would like to mention? 


Thank you for taking the time to do this today, a lot of valuable information was collected. You will all 
receive an invitation from me for the opening of the exhibition. I hope you will come and see what the 


end results of this study are. 


STUDENT QUESTIONNAIRE 


This questionnaire consists of 17 questions on four pages. When asked to respond to a statement, please in- 
dicate whether you strongly agree, agree, neither agree nor disagree, disagree or strongly disagree. Please 


explain your answer whenever appropriate. 
Preferred Methods of Instruction 


1) Anumber of learning methods are employed in the teaching blocks from first and second-year 
studies in medicine. Based on my preferred style of learning, I find each of the following learning 


methods very effective. 


Strongly Agree Neither Disagree Strongly 

agree A nor D disagree 
Lecture 0 i) O O a) 
Laboratory O O 0 O 
Small group tutorials O O O O O 
Self-directed learning exercises a) O O O O 
Interactive computer lessons O O O O 
Clinical case scenarios O O O a) a) 
Other (specify: panel 0 O O O 


2) Rank your preferences for the teaching methods employed by the instructors in each of the 


teaching blocks from first and second-year studies (1 indicates most preferred and 10 indicates least 


preferred). 

____ Introductory block ____ Gastroenterology and Nutrition 
___ Infection, Immunity and Inflammation ____ Reproductive Medicine & Urology 
_____ Endocrinology ____ Musculoskeletal System 

____ Cardiovascular, Pulmonary and Renal ____ Neurosciences 

____ Patient-Centred Care pees Oncology, 


Briefly describe why you favor the methods in the block you prefer most. 


3) My understanding of the content covered in the Gastroenterology block is very high. 
Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree O 
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4) The level of difficulty of the content of the case I just covered as part of this research study was on 
par with the material covered in the Gastroenterology block. 


Strongly agree 0 Agree 0 Neither A nor DO Disagree 0 Strongly disagree 0 


Approach to Patient Cases 


5) When I am provided with a patient case in class, I use a particular method to analyze and develop a 
hypothesis about the patient condition. 
Strongly agree O Agree 0 Neither Anor DO Disagree 0 Strongly disagree 0 


Explain: 


6) I think it is important to use a method or framework to analyze and solve a patient case. 


Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree O 


Explain: 


7) Ithink the diagram construction activity in this research study was helpful in allowing me to analyze 
and understand the patient problem and move closer to a differential diagnosis? 


Strongly agree 0 Agree 0 Neither A nor DO Disagree UO Strongly disagree O 


Explain: 


8) 


9) 


10) 


11) 


I think the activity that was just completed in this research study helped my group understand 
aspects of the case that we may not have otherwise realized. 


Strongly agree 0 Agree 0 Neither A nor D O Disagree 0 Strongly disagree O 


Explain (provide examples): 


I think the activity of constructing diagrams of a patient case, similar to what was done in this study, 
is a good way to prepare for a small group discussion with an instructor. 


Strongly agree O Agree O Neither A nor DO Disagree UO Strongly disagree 0 


Explain: 


I think the diagram construction tools provided for the task exercise are easy to use. 
Strongly agree O Agree 0 Neither A nor DO Disagree 0 Strongly disagree 0 
Explain: 


I would you use a similar process to solve patient cases as a group in the future. 


Strongly agree U Agree 0 Neither A nor DO Disagree O Strongly disagree O 


Explain: 


Personal Use of Technology 


12) 


13) 


14) 
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17) 


Iam very comfortable using computer technology (web browsing, e-mail, word processing, etc.). 


Strongly agree 0 Agree 0 Neither A nor DO Disagree 0 Strongly disagree fs] 


During my leisure time in a typical week, I estimate I use the computer: 


Less than 1 hour 0 1 to 4 hours O 5 to 8 hours O 9 to 12 hours O 13 or more O 


During my work/study time in a typical week, I estimate I use the computer: 
Less than 1 hour O 1 to 4 hours 0 5 to 8 hours O 9 to 12 hours O 13 or more 0 


In a typical week, I frequently use a computer for the following: 


Strongly Agree Neither Disagree Strongly 
agree Anor D disagree 
E-mail a) O O O O 
Instant messaging O O O O O 
Forums /Conferencing/Blogs O [I | | O 
World Wide Web surfing O o O O O 
Computer-assisted learning O O O O O 
Word processing a 2 Oo O O 
Games O O O O O 
Other (specify: ) Oo Oo Oo O 
Rank the location of the computers you use to access web sites and e-mail with five being the most 


frequent and one being the least frequent. Only number the spaces that apply to your habits for using 


the Internet. 


Home Computer on the wards 
Campus computer lab Friend’s home 
Other (please specify: ) 


How do you most often connect to the Internet? 


Dial-up modem 0 High-speed Internet 0 Wireless 0 None 0 


STUDENT FOCUS GROUP PROTOCOL 


Context 

As I stated earlier, the purpose of this study is to collect student and instructor feedback on the de- 
sign of a method to supplement small group, problem-based learning groups in the Faculty of Medicine. 
Ultimately, our desire is to create a web-based application that would allow you to construct these dia- 
grams as a group. I’ve brought some mock-ups of what this application might look like to show you and I 
have some questions to ask you about the exercise we just did and about this concept. We’ll get to that in 


just a few moments. First I wanted to show you what this program might look like: 


Figure 1: Login screen 
This screen would allow you to sign-in with a unique user id and be automatically assigned to a 


group, depending on the teaching block you happen to be in. 


Figure 2; Course and case selection screen 
This screen would allow you to select the teaching block you happen to be studying in and select 
the case that you've been assigned to work on. You will also be able to review your diagrams and the 


diagrams of other groups from previous exercises. 


Figure 3: Case presentation 
This screen shows the patient presentation with a text description and a picture. Blood tests, X-Rays, 
EKG readouts or any other diagnostic results could be included. Video and audio components could 


also be incorporated. 


Figure 4; Diagram construction interface 
This screen shows some suggestions for tools necessary to create a diagram on the screen in a similar 
fashion to what you have just completed with paper. You can create a node, label it, fill out important 
information, write a rationale and tag the node with data that could be helpful for you to organize 


yourself. 


Figure 5: Search and sort 


This screen shows how you could use the properties to sort the diagram. 


I wanted to ask you some questions about your experience creating the diagram and 


about some of the things I’ve just shown you for a web site. 


Questions 

1. How did you enjoy the activity that you completed earlier? Why or why not? 

2. Do you enjoy small group or problem-based learning sessions as they currently 
exist in the teaching blocks? Why or why not? 
Do you have a method of approaching clinical cases? What is it? 

4. Do you think a method like the one used today was a good way to prepare for small group 
discussions? Why or why not? 

5. After completing this task exercise today, do you think it made you think differently about the 
way you might approach a patient case? How? 

6. Do you think a web-based resource to prepare for small group discussions, something like 
the one I showed you, would be beneficial? How? Why? 

7. Would you use a web resource such as the one I showed you? Why or why not? 

8. Would you create the diagram sitting together in front of a computer or would you prefer to be 
able to do it as a group from remote computers? Why or why not? 

9. The web site would contain many of the features of the paper tools that you used today in 


the activity — labelling, rationale, description, relevance, priority, etc. Did you find those 


features useful? Why or why not? 


10. What additional features or tools do you think would be useful in such a web site? 


That is all the questions I have for today. Does anyone have anything they would like to mention? 


Thank you for taking the time to do this today, a lot of valuable information must have been collected. 


You will all receive an invitation from me for the opening of the exhibition. I hope you will come and see 


the end results of this study. 
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TESTING MATERIALS: Diagram construction tools 


Fact/Title: 


Rationale: 


Physical Exam 


Fact/Title: 


C) Rationale: 


Fact/Title: 


Rationale: 


Patient History 


Fact/Title: 


oO Rationale: 


Fact/Title: 


Laboratory & 


Rationale: 


Fact/Title: 


Diagnostics 


Rationale: 


tesponsibility 


priority 


TESTING MATERIALS: Example physical exam and test result cards 


Abdominal Exam 


Enlarged, tender liver, palpable 


No splenomegaly 
No ascites 

Bowl sounds normal 
No other masses 

No Courvoisiers’s sign 


No Murphy's sign 


(16c1n vertical span at mid clavicular line) 


Physical Exam 


[Abdominal Exam] 


Liver Function Tests 


PT 
NL Range 
AST [SGOT] au/t) 470 
10-40 
ALT [SGPT] (u/L) 420 


Complete Blood Count 
Result Normal 
7 G 
Laboratory & Me oc 
Diagnostics \ 
[Complete Blood Count] eo. f} 
@ € cA 


Serology 


Hepatitis A Anti-HAV 
IgM 
++ 


Hepatitis B HBsAg 


HBeAg Virus Anti-HBc Ant-HBe Ant-HBs 


++ 


HepatitisC — Anti-HCV 
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TESTING MATERIALS: Example differential diagnosis matrix 
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TESTING MATERIALS: Responsibility, priority & urgency stickers 
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{first five minutes} 
So what did you think? 


I liked it - I thought it was fun - I’m a visual learner and thinker so it was nice to have that 
structure - | thought it was helpful. 


So you guys know you’ve got a case right? That the guy has Hep A. Any hypothesis on how he got 
it? 


Fecal/oral transmission. 

He’s a med technician so I was thinking at work maybe. 

Because there was no travel. 

Its pretty contagious, so.... 

You can get it from restaurants. 

But it wasn’t from his family, so it must have been outside of the home (work, restaurants)? 
According to the case the way that its written, it was brought home by his daughters who were in 
daycare. So here we go - its a public health situation and what you would do as a physician. So 
you didn’t get thrown by his response to the sexual limitation thing? 

If it was Hep C or something I’d be more concerned about his past sexual history. 

He could be into some weird sexual practices. but that’s probably not it. 

I was trying to visual what it would look like on the computer- I think if there was a program 
that had these types of headings and you could navigate through them and they could be kind 
of like things you could fill in - it would be easy to use. 

Do you think it would be useful? I want to talk a little about your prior experience in this sort of 
thing before we talk about the website. I think, from the way that I’ve thought about this, you 
guys did it the way that I imagined it happening. Do you have some sort of experience with 
doing this, maybe not in a visual way like this, but going through cases like this? 


That’s how they teach us to do it. 


We kind of put headings on the board, and put, for example “what was the chief complaint” 
and then try to break it down that way as well. 


Do you want to do something we did really differently? We forgot to do the differential before 
we started ordering lab things because normally we have a big list of things it could be, with 
the most likely things at the top and then usually with the cases, the way they do it, you order 
the tests and then you flip to the next page, and then “here’s the results”, so you start crossing 
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things off and say “Oh - this points to this one a lot” etc. and then you end up with one. Like 
with the Hep A we would have ended up in the same place. But we would have had other stuff 
on the list. But when we got the Hep A result, it kind of rules out everything else pretty much. 
The only difference between how we did it this time and how we normally do it is that we 
should have had the differential first, but the process was still the same I think? 


Can you use the differential to use which lab tests you might be interested in doing, because 
otherwise you might think “well maybe its this” and its very inefficient, and we are taught to 


think about what the cost of tests are, 


I think this would be a good format for group work, but if you’re working by yourself it 
actually slows you down {all agreed} 


To come up with the headings - I found it was hard to know, e.g. what’s a large node, what’s 
a small node, and then even going on from there it was hard to know what headings to use. It 
doesn’t really matter, but if there was {discussion}...... 

But then it kind of gives you a clue, you have to do less work if the headings are there. Its 
good for us to do the headings but its hard for us to know sometimes what the structure was 


that you needed to use. 


Sometimes its nice to think it through yourself. Some people like to throw out answers before 
you even know what the question was. 


So this allowed you to think about why and reason through it. 
Yes, to think it through yourself. 


What about having a visual format where you can see what you've done and what you haven’t 
done? Did you use it that way? 


Yes 
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{second five minutes} 


I tend to be more scattered in my thinking. I still haven’t got a rigid framework, so just to go 
through things logically and make sure you haven’t missed anything - it was nice. 


So this helped organize your thinking in the process and in the group? 


Normally I would write this stuff down, and I still need to write stuff down so I can see it. | 
may draw a little flow-chart so I guess I am kind of doing it visually. Its hard to just do it in 
my head. 


Do you think this may have been too basic for you guys at this level? 


Well its the same kind of cases we’ve been using all along, so it wouldn’t say it was harder or 
easier. 


So its probably the right level and mix and blend of things? 


Well there were some things we still had to look up. In the GI block, I think it was fair. I 
guess when you get to Hep A - its a lot easier to diagnose than some of the other things you 
could have made it. When we get into cancers and things, there’s not always a smoking gun 
like “oh that’s what it is” 


With the information that we got from you that we had to ask you - how would that show up 
on the web? 


That might be a mechanism where you would have to make a request and it could be set up 
where it would release it in a time frame, and certain things would start popping up and say “no”, 
“yes”, “no”....etc. so it may be you request that and there might be sort of a realistic turnaround 
- like it takes 24 hours to get a lab results so maybe the program could be set up to release those 


and in 24 hours the labs would show up. How would you guys feel about that type of thing? 

I think it would work. As far as asking questions about physical exam and such, is it possible 
to have real time - instead of having 20 preceptors running 20 groups you could have one 
person at a computer for the time that the group is meeting. E.g. type in a request and then get 
a real time feedback from someone that could just be in their office. 

That’s a possibility 


Or it could just be stored in the computer. 


For example, an immunization history, that’s something you don’t need a test for - you could 
just ask the guy. 


Is there some way to make the program so that it recognizes key words? 


Yes, or there could be selections - maybe a standard set of selections for every case. 
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I think that would work better. 


One way of thinking like using a web based application for this is exactly what you say - instead of 
having 12 different preceptors for 12 different groups, you could have one preceptor for all 160 
students. 


That could get really crazy if you’re bombarding messages at the preceptor... 


You could have it without the questions having to go through - if you were to have information 
stored in the computer that the preceptor would tell us, then we could just pull it up by 
pressing on a button, but you might require something that somebody else has already filled 
in and submitted before we would get that information 


The keyword thing is good. It would just be more work for you to have to have all that 
permission in there at the beginning with, e.g. immunization records, past sexual history, 
medical history, etc. 


If its all given to you, for example, the physical exam, we have to know to ask if there is a ¢ 
instead of saying “can I have the GI exam and getting all this stuff. its kind of given to us more 
so then we’re actually knowing that we have to specifically look for this because this is 
important - because that’s how its actually going to be when we see patients. I think this is too 
“handed” to us because then its good for the PBL and the book smarts but it doesn’t prepare 
you for the real situations. 


{third five minutes} 
Yes. Its like a blank page - you really know nothing except maybe their age and their chief 
complaint - that’s what its like in real life. Sometimes on he PBL’s I find, they give you all this 


information, but sometimes they kind of spoon feed you all the important findings. You can 
totally make a diagnosis by reading. 


Yes that was my concern here, if I gave you all the physical exam and all the laboratory results 
right off the bat you’d go “Oh its Hepatitis A” {laughter} 


On a lot of PBL’s they'll go through all the systems and tell you not everything you saw but just 
the things that would confirm the diagnosis. 


Or they’ll give you a bunch of useless information and the last sentence would be “has rings on 
his iris and his copper’s too high”. 


Too specific - like you can make a diagnosis from just on those things...... 
If you were actually examining this guy in real life would you see those rings, would you know 
to look for them? Would you be able to make a diagnosis? An ophthalmologist would, but 


would we? 


You would do so much other stuff first - so probably. 
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I liked having to ask - if there were some way you could incorporate that in - don't just give 
them all the information - some way they would have to request it - however you want to 
incorporate that - by keyword searches - that type of thing. 


Or if you give it the results - also give not just the stuff that confirms or is against the diagnosis, 
but give them all the information. E.g. I want to examine the abdomen, here’s all the 
information from the abdomen, now sift through this and find out what’s relevant. 


You sort of did that with the car - like you had a lot of negative results, which are important 
too, Like the CBC for example. I go on some of the PBL’s and its like you order a CBC and it 
comes back with “white blood cells are elevated”. So they’ve already filtered out the important 
stuff for you - they’ve kind of done half your job. 


So they don’t give you actual results? 
Sometimes they do, sometimes they don’t. I prefer it when they just give you a big glob of 
information - the kind you get back from the lab - then its up to you to decide what's 


important - I think that’s the best way. 


It was nice of you to give us “normals” because they come back from the lab with normal 
values right? So that’s the way it really is. 


It would be different if we asked for a Liver Function Test - it could give us the liver function 
test - so that’s a way of being a key word “liver function test” then you give it back to us. For 
example a physical exam - we could have asked “C’s sign”, and then with that key word it 
could give it back positive or negative, instead of just giving it to us, but we don’t actually 
know it, but we recognize it when we see it. Its like the difference between doing a long answer 
test and a multiple choice test where you actually have to recall the information yourselves. 


Excellent. Now what about the group interaction? Have any of you had PBL’s to interact with 
one another pretty much the same way you did here? 


It depends who’s running it. 

Sometimes there are people who are really dominate the talk. 
Sometimes its the professor - they just lecture. 

So what are your thoughts on using this versus what has happened here? 


I think groups of four or five wouldn't be a problem - I think they’d be able to work together. 
I think its something you have to do in groups in front of one computer. 


I guess the only issue is that how would you get 160 people in one computer lab at the same 
time going through this and not overhearing each other. The logistics would be really hard. 


You couldn’t do it in the library or other computer labs because other people would be 
working. 
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If there would be six people around one computer, it would be kind of annoying I think 
because you wouldn’t be able to see. 


I don’t think going home and going it individually as a group would ever work. Some people 
would leave it to the last minute - others would want to do it immediately. 


{fourth five minutes} 


What if there were some sort of collaborative tools, e.g. instant messaging feature or email 
feature - where you could email each other, you could agree? 


That could work. You’d have to agree on a time. You'd have to be there working at the same 
time, but instant messaging is so cumbersome. 


I guess there’s also a problem for people that don’t know instant messaging or don’t type fast. 
I think part of the nice thing about this is that we were talking and discussing things before we 
immediately wrote things down, so we had to think about that before we actually decided on 
something that was important that we should put up. 

And although we didn’t talk much, I think if you took the amount we said and tried to type it, 
it would be a lot. With instant messaging you have to keep things concise and to the point, 
whereas here we had a chance to bounce ideas around. I think instant messaging would be too 
tough. 

How many people have laptops that they regularly bring? 

10 percent? 

Is there wireless in the building? 

There is wireless. 


People with laptops having really caught on that much. 


So once the logistics of how big a group would be and how you would decide to go about this as a 
group, do you think you’d use something like this? 


I think if it didn’t double the work - like if you didn’t have to meet in small groups again - and 
if the groups were small. 


What if you were divided into sub-groups in your small groups, and you guys did this at home, 
and then you came to your small group and used this to present - e.g. this is a diagram that we 
did and this is the result. Then the PDL meeting with the whole group would turn into a 
presentation of what you did - sort of a wrap up - do you think that 

would be too much of a duplication? 
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Do you mean if we each did it individually? 


No if there’s 15 in your group and there’s 3 groups of five, and then the 3 groups came and 
shot their diagram and threw it up on a projector and talked about it - give a presentation back 
to the group. 


Its kind of hard because at the end of this one, for example, you'd end up having to listen to 
three different groups talking about why it was Hep A. I think everyone would come down to 
the same conclusion. 


Its pretty frustrating sitting listening to all the other groups because other groups would have 30 
points on their slide - and there would be one thing that was different. 


{all agreed} 


It would be nice to have a wrap up session - like for some cases there are 3 sessions - what if 2 
are spent doing this and the last one would be spent with a preceptor where he could just 
cover some key things, cover questions and make sure we all have the same conclusion. I 
think a presentation would get redundant. 


{fifth five minutes} 


These are sketches on what I imagine the site might look like. {explains what the site would look 
like and how it would operate} It would be a blank page. Tabs: Navigate, Diagram and Data and 
a Zoom feature. You can click on a tab that would pull up a palette which would let you create 
notes and then its got some pieces that talk about relevance and responsibility, priority and also 
allows you to add more properties. If you want to create a note, drag and drop. Once you do 
that, you can open a palette that would allow you to type in information. Enter in the title, facts 
about it and the rationale and you can hyperlink to the internet. You would keep going on the 
diagram - i.e. chief complaint - nausea - could create a justification for that. So youve created a 
whole case. You can zoom in with the Navigate feature. There might be other features - you 
want to print this out and present to the preceptor as a linear document. There would be major 
headings, sub headings and sub-sub headings. Editing would be afeature. I think from what 
I’m hearing from this group is that you like the diagram approach. You can either print the 
linear or the diagram. 


Personally I think its okay to do it that way the first time, but in the end I'd like to have a linear 
printed. 


You can put in a bullet for differential before even your labs, and make a bullet for them, so 
people can make a list of what they thought it was in priority. 


Yes that’s something that I’m missing on the interface is the differential diagnosis matrix. 


Even if you come up with a way to link the test result to your differential, just to show what 
your line of thinking is - like what differential leads to which test, etc. 


So you’d like to associate what you add in there to your differential? 
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Because once you’ve seen your lab result, then you can go back and eliminate some of your 
differential. 


Because some of the tests were for one specific thing. 

{sixth five minutes} 

That’s really good - that’s something I hadn’t thought about. The nice thing about the power of 
the computer is that if you guys decide to sketch all this stuff out, then you fill in some of the 
information, if you can’t remember what is yours, the computer allows you “select ability” - if 


you want to see only your own, high priority, etc. 


So a group would have access to all the same stuff - if | made any changes at home, Scott could 
look at it later. 


Yes - only your group though. 


I don’t know how this would be structured, but it could compare diagrams across groups - i.e. 
when comparing one diagram to another, you can see differences. 


In this case how much time is spent on patient history and it shows there. 


P’ve had a lot of discussion with faculty and they was really excited. Odds of a successful 
diagnosis increase exponentially. 


In some cases they give you things that you would have never had any idea about - so you 
have to come back and research it. That’s when you would use the other stickers. 


We put the stickers on at the end seems more artificial to me - like we were just using them 
because they were there, but if they weren’t there - I wouldn’t have thought of it. Maybe that’s 
a line of thinking that we should be developing - things that are more important / less 


important. Could help with prioritizing. 


I suppose with patient history, you could rank the physical signs from how important you think 
it is, but in the end I don’t think it serves any importance to rank them. 


You might if there was a lot of extraneous information. 

What about multi-media components - would you enjoy hearing heart sounds, seeing patient 
interviews in video format, seeing animations that might explain something, x-rays, ultrasounds, 
CT scans. 


{all agreed} 


{seventh five minutes} 
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You could show us an xray and we'd have to interpret it ourselves - that would be great. 
They’ve done it a bit on the PBL - they’ll print off an EKG reading. 


But other times it has a heart gallop - but I’ve never heard a heart gallop. 


Sounds would be nice. Patient interviews I don’t know so much about because we have the 
physical exam, or pictures of things that you talk about - like something obscure - they might 
want to see it. You could do long sounds, etc. 


Anything else I haven’t brought up? 


If you were going to have groups that do this and then have people come back together in a 
PBL, I would make the PBL’s very short that you would come back for about an hour 
maximum wrap up. 


You mean the length of the small groups at the wrap up? 


Yes, if people were doing this on their own time and then meet with the preceptor at the end. 
but it wouldn’t be necessary to be a long session. 


Yes this would get rid of 3 hour PBL sessions. This time could be used more for self-study. 
That’s another issue - do you guys enjoy self- study or do you feel pushed out of the nest? 


This is structured enough so we’re not just having a paper case that we need to do on our own, 
and it kind of forces us to do it because there’s a deadline. Its still problem-based learning 
regardless of how many people are in the group. 


So you like to take ownership for your own learning, rather than relying on having a tutor there 
to give you constant feedback to you? Do you like this idea of going out and covering it on your 
own? 


I think there’s enough guidance in the program - so its okay. 


Personally I do think I’ve learned from PBL - but the doctors that volunteer their time for that, 
they only have so much time over the course of a semester that they can donate. I would get 
more value out of it if we could get more shadowing sessions with the doctors. I get so much 
more out of that. If by having less 3 hour PBL, we could incorporate more actual clinic 
experience, it would help me a lot. We used to go around listening to patient’s hearts first 
hand instead of reading about it. You were talking more about using time more efficiently I 
think that would be good way to do it. 


I find seeing patients one of the most helpful things. 


Some of the faculty members I talked to thought it would be a great way to get residents and 
more senior students to fill this out, so junior students, if they have the time, they could look at 
what cases other people are dealing with, they could. You guys would have access to that 
information. Do you think you might use it? 
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As far as budgeting my time, it would be way down on my list. 


It would be useful, but would take too much time. It would be nice, but I don’t think we’d 
have the time now - maybe in third year when we're actually on the wards. 


{eighth five minutes} 


Do you know what? portfolios are? I think when you graduate and get your license do you 
have to have a portfolio of patients that you have dealt with or something like that? I’m not 
sure if you do or not. 


I know surgeons have a thing like “I’ve done x numbers of appendix removals”, numbers of 
surgeries, etc. but haven’t heard anything on the medical side. 


Were you thinking of using this type of format? 


If some I know is interested in your work, and wanted to see what you ve been doing with patient 
cases, you could potentially show them something like this. That was another idea that popped 
up from a faculty member, but I’m not sure - I'll have to inquire more into that. 


So let me get a feel for your “burning desire” to use something like this. Are you really 
impressed, you’ve “got to have it”, or is this “this would be nice” or is it “this was a fun exercise 
to do today but I don’t think I'd ever do it again”? 


It would be nice and if it was done properly it would be useful, but as far as a critical learning 
component, I don’t think so - but I think there is a better way to do PBL than how they do it 
now. 


Do you guys agree? 


I would like to do this in lieu of, like one or the other, but as long as it doesn’t become “extra 
homework” then I think it is a good idea. 


Extra homework - meaning, like “in this block instead of doing 4 PBL’s we’re going to do eight.” 
Is that what you mean by extra homework? 


If they schedule lecture instead of PBL time - so this just becomes one more thing to do that 
we're not giving any scheduled time to do it. If this is what we do during our scheduled PBL 
time then its fine. 


I can imagine that this would not replace, but use time more effectively in PBL and not add to 
work that’s already there. Thanks for your time and I really appreciate it. I got an amazing 
amount of information out of this. You guys had some really great ideas on how I can 
improve it. Hopefully one day this will be your website. 


-END- 
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{first five minutes} 
Did you think this exercise was useful? You can be brutally honest. 


Well it depends - are you trying to run this as a group session or kind of an independent 
internet problem-based thing. 


That’s a question I might want to hear feedback on. That’s a possibility - it could be set up so 
that you meet together as a group - its however you guys really want to organize yourselves. It 
could bea distance collaboration type of thing, or it could be where we say “okay we’re going to 
meet at 3:00 at my house, etc. etc.” so those are all possibilities. Which one of those possibilities 
sounds feasible to you? 


For me, if each of those big blocks that we put up - if it became like a form that you have to fill 
out, then - if the answers could be provided - I don’t even see why you'd have to be in a group 
at all. But then if you have it in a form format - and you start filling it out and its kind of 
interactive...., I think we’re kind of taught that you can make your own approach to history as 
long as you keep it consistent. If you force an approach on us by making it kind of like a 
form... I don’t know if that’s bad or not. 


The thing is, we already know the stuff technically, sort of. If we were to do this again, we'd get 
stuck everywhere- every single thing would be a learning issue. 


I also think its hard because you don’t know what the exact format should be, so we kind of 
made that up, but someone else might do it differently and I think we got sidetracked on 
discussing what should be a big thing...etc. so there was too much of that and it added a lot of 
time so if there was a very specific format - here’s a history of present illness - put all the stuff 
in there....it would be much easier in a group, but otherwise it would take forever as a group. 


I don’t know if you saw us but we were arguing over body temperature. Its a small detail that 
could be really important or not at all. 


So part of this is process, and how much have you learned about the process of treating a patient 
as far as the steps you would do: family history, social history, etc. ¢ 


They ’ve just started teaching us that, I guess we still have a long way to go. 


I would really like to leave it as an open and flexible tool where you can decide what you want 
to put in the patient history and not a form, because | think if its a form, its almost like you’re 
just clicking on these little nodes and you're not really learning through going through a 
process. 


I think if its flexible it would be more useful if you could do it yourself, and then come in and 
you could discuss why you put certain things in a certain area. 


Would you do it in such a way where you’d say “okay I’m going to handle this part of the 
patient history” or would you prefer to do different elements on your own and come back and 


meet as a group? 
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Are you going to run the sessions so we already know the stuff that is going to be covered at 
the lecture and we’re simply reviewing? 


Yes - well its sort of an integration on what you learned at lecture with what you’re doing. 


Because we had something in GI block - Survivor Challenge - we'd get a question a lot like 
this, except it would be independent and we'd have to go research it by ourselves. But they 
made the cases really tough so you really had to read ahead. You’re not going to do 
something like that are you? 


The case that I have there I got from another medical school in Ohio, and that’s one thing that 
they do is an intro to the GI block. So the idea is - you go to the lecture, you learn about 
jaundice, pre-hepatic conditions, hepatic conditions, post-hepatic conditions, and then this is 
an opportunity to apply what you’ve learned to a patient case. What may happen is that you 
sketch out a framework and then you go meet and say - okay these are the issues that we talked 
about and I think its this, and we thought this was urgent, we thought this was important, and 
these are some of the highlights of the case. And you might even be able to use a diagram to 
present this to the group. 


{second five minutes} 


I don’t see how this is different from just getting the problem on a sheet of paper and then 
discussing it and then coming back to your group. I’m trying to figure out what this program 
can offer that would make things more organized or easier or interesting. 


What you’re saying is we can do it from home, on line, and on our own time, right? 


But they could hand out the case during class and we’ll take it home, read the case and make 
an outline. 


Why don’t they do physicals the same way? Why don’t they just use it horseback? 


I think this might be easier, because if they have stuff where you have to complete a section 
before you can move on and get the extra information, it might be easier because when its just 
on a separate sheet and half the time they've got for example, “pulmonary” and then they stick 
the answer right after the question. No challenge. What would I really do if a six year old 
came in short of breath? 


Are they going to make this mandatory, because that would make a huge difference. 


This is not intended to be - right now there are issues - some people in some blocks aren’t doing 
any PBL. And other blocks are doing PBL but may be doing it badly. So its a way to standardize 
it and not add toit. They would be doing four PBL sessions in GI now, there wouldn’t be this 
four plus another four on line. This would be sort of a supplemental tool and a supporting tool 
to PBL, so like alot of PBL sessions, the first thing you do is go in - and the first hour is 
administrative - talking about vitals and pt. history, etc. The second or third hour is when you 
really get into the meat of it - or it could go for days. The first hour may be that you’re 
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presented with a case, discuss the case and go off and do some work on that. The intention is 
that this tool would reduce the in-class time and maybe the face to face time in that initial set up 
phase, so that you become educated and prepared for the face to face time with your instructor, 
where the really useful learning goes on. Do you think that this tool might do that? 


I think, if it is like a supplement to your learning. In our MSK Rheumatology block we have 
those times where we meet with just an instructor and they guide us through our case. I found 
it very useful because they would go into the significant parts of the case - they would point out 
why things are significant, and pretty much give rationales for everything. So as a summary - if 
you tried to do this case by yourself and you get stuck - they would have an answer system, 
where you could go through each point and find the rationale for each symptom or presenting 
condition. I find this very useful as a learning tool versus just in class hearing someone talk. 

It sticks in your mind more. As a thing for a group to work through, it seems a bit time 
consuming. When you do have PBL stuff you have a sheet and you underline the important 
things, you can recognize a pattern and you can come to a differential and final diagnosis 
without having to list out each point separately. 


{Third five minutes} 
Do you find the tools confusing, and the nodes and all that kind of stuff. 


The nodes? We just had to get it straight in our heads what was important, what was the sell 
point. As far as the tools I guess we didn’t really assign it until the last 10 seconds. If we got 
used to it, then it probably would be useful to organize information once you had it all down, 
but we were still trying to assign what the symbols meant to us. 


This wouldn’t replace PBL would it? 
It would not be an entire replacement. Its a means to use time more effectively. 
I found our PBI instructor would lecture for the full 2 hours and we wouldn’t do any talking. 


So in that respect, what do you think of this sort of arrangement? You guys did all the talking, 
right? 


Yes. But some guidance is good too, 


One of the things I like about PBL is that some of the professors don’t like you to prepare 
beforehand. That’s almost more interesting that way because then you really have to think on 
your feet because when you're at home and you can, you look up this stuff and you know what 
the things are, so you have all your textbooks but you won’t necessarily in the office - you will 
take that time afterwards, but you have to get through it yourself at first. So I like that basis and 
then afterwards you assign learning objectives and come back later and discuss what you’ve 
learned. I think that clearly identifies the gaps in your knowledge. 


I know in Calgary I think they have some program that they go through that goes by systems, 
so its like - you choose one point like cardiovascular - and then all of a sudden this huge web 
of associated problems or symptoms comes up, and you just keep going to each one and it 
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keeps opening up another web. I think that’s a very good tool that they use to supplement 
class material. 


Yes, they have scheme space approach and those are supporting schemes. I don’t know too 
much about that. 


I’ve only looked through one of them once, and I thought it kind of did something like this, 
not with a case, but as a body system based typed of thing. I think it could definitely safe a lot 
of time, depending on how its implemented and used. 


I think that scheme stuff gives you a methodology for approaching certain things. Its more like 
youre developing the methodology on your own. What do you think about that approach? Do 
you think its important to develop something on your own? Or important to have that 
framework given to you? That comes back to what you were saying about the form. 


We were given a form once last semester. We had to do a history on somebody and then hand 
in this form that we basically filled out, and that’s not going to do anything to help me 
remember what to ask somebody when I’m on the wards. So its all about me forming my own 
pattern and then getting to this routine of remembering all these steps to go through, so I think 
its much better if we form our own, because then we can try different ones if we want to see 
how well it works for us. 


I think if you get exposed to a whole bunch of different ones, though - you kind of choose 
your own. I remember in triple I - one of our professors had a four base one and stuff and it 
was nice because it was like, “History of present illness” “history of past illness” “social 
history” and “family history”. and then I can remember the four of them and I find it a lot 
easier for me to work that way, instead of the 80 things under history of past illness. 


But once your pattern is forming, you kind of just stick with it - its more or less to keep 
reforming that same method once you already have a certain pattern down. 


And this is intended for a period in medical school when you are forming that process and that 
pattern. But it could be set up where you establish a pattern and save that pattern and call that 
pattern up every time - sort of a template type of approach. But you create the template on your 
own. Its not forced on you. 


That would be nice. 
That could be really good for early medical school, when you have nothing at all and to help 
get something like that early would be a real advantage. I don’t know about you guys, but it 


took me a long time to get something like that going. 


Or maybe it could offer different templates that you can just choose one to see which approach 
fits more comfortably with the way you think, and then you can modify it because it could just 
be general. 


{third five minutes} 
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After you take the time to personalize something, it makes it a lost easier. 
I noticed that you weren’t using the differential diagnosis very much. There was a part up there 
where maybe you thought it was Hep C. Did you know at a certain point as you are formulating 


this process where you would use the differential diagnosis? 


PBL cases usually are structured like “what are the most important symptoms” and “what is the 
differential diagnosis” for those. So you did it almost right initially. 


We couldn’t give you an appropriate answer because we were still ranking all the symptoms. 
So it wasn’t until the final minutes that we actually ruled out. but as we went through, definitely 
everything popped up - a whole bunch of differentials popped up but as we went through we 
ruled some things out so you don’t have a completed list till the end. But a differential can be 
everything. Maybe we didn’t make use of the tool because we weren't sure how to use it. 

It was also this “vindicative”. I’ve never heard of that - it made me not want to use the form. 


Vitamin C. 


It would be useful - when you’re trying to think of a differential you don’t always go through 
the possibilities. 


Maybe if you had a separate pod or something or something that would bring it up right then 
and there so you don’t just kind of go on and see. 


Were you at a point now where you were going to develop the differential? 

We got the results for the lab, too - so that totally screwed the differential. 

Usually they don’t give us the lab results until we come up with the differential. 

Once you get lab results its too easy to cheat. 

You already have to have a differential in order to decide what you're going to do in the 
physical exam and the lab. So its well before.....we were e all thinking the same things - it was 
coming out of our mouths, but we didn’t necessarily write it down. 

So - let me get a consensus here. Would you use this method again? 

If it was set up on the computer, yes. writing it down takes too much time. 

I think it would be more useful for a first year, because we’ve been through two years of PBLs 
so we already know what everything is. It would be very useful in the beginning because when 


were first looking at those PBL cases we had no idea how to organize it. 


I don’t think everyone will find it useful. I’m pretty sure if they find it useful in first year they 
will continue to use it. Just depends on how people like it. 
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I’m asking you guys because you have the experience in this, and I’m trying to figure out where 
this fits, and its appropriate in the beginning or if its appropriate now or.. 


I know ina lot of other courses, where we do the PBL on the stuff before we have the lecture - 
that would be really useful for that because its good to organize the way that you think for 


these things. 


Is this sort of the traditional approach that you do to PBL - patient history, physical exam and 
laboratory, or do you look at it in different ways? 


There are always guiding questions in our PBL so we don’t really initiate things that much. 


We don’t go and specifically build a history then build a physical -- we just look at it and go 
from there. Just pull out the main points and the rationales. 


{fourth five minutes} 

What would you think about doing a different approach where its like “want you to look at the 
path of physiology” for example. How does the pathology, the anatomy and the physiology 
relate to the symptom?” 

I need more guidance for something like that - because you get that statement out to you in 
lecture - but its easier if you have someone there that says “this is how this interacts with this” 
so I wouldn’t find this useful at all on the computer. 

{fifth five minutes} 

If there was a link to information on the computer within the program, then it could work. 
Because in lectures they try to have one disease and then try to do all the lectures on that issue, 
but the information isn’t really summarized - the lecture is all in one day so you don’t really 
keep track of it all. If there’s a visual summary it would be useful. 

If you could do that it would be really helpful in Renal Block, just because there is so much 
pathology and no one really correlated that to physical symptoms - I think that would be really 
useful. 


Do you have to memorize the pathology? 


Yes - but its disjointed, i.e. certain diseases present this way but you don’t correlate that to..,.if 
you had a slide next to the symptoms, it would come in handy. 


That’s something that we would like to use. Build that asa histology/ pathology type of thing - 
in the future. 


{Explanation of how to use the program}. 
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....'o correspond to the diagram there might be a linear document. One of the exercises we get 
you guys to write is a case report. So this diagram might help you write a case report. What is in 
the diagram will correspond to the linear document. We have patient history - the big node, and 
then social history - he’s got 2 daughters, and 2, then you write the rationale. and maybe link to 
internet resources, That would allow you to print it out and make a copy for yourself, or send it 
to your instructor if they ask you to do that. You can add stuff here and it corresponds to what 
happens in the diagram. The two are connected. Then what you could also do is compare 
yourself and talk to other students that thought that it might be interesting to see what other 
groups are doing. 


Are you able to access the different groups? 
Yes, after the fact. 


Would this be something - would you have access to a laptop to bring here for PBL groups. 
This would add a lot of extra work if someone has to go home and type out what everything 
that we thought of - because there’s always one person writing everything down in PBL - so if 
you just get to sign out a lap top and bring it back that would be good. 


That would be one of my recommendations that the library reserve 5 or 6 laptops. I know that 
there are plans to make the fishbowl enabled with wireless and I hope that will also increase the 
signal. 


That would be really useful to do it as a group here and check on the internet later on. 


The other thing is that you would probably get test results and more patient history by keyword 
search. What I gave to you is a big chunk of information all at once, but you could ask for 
information just by typing it in and saying “submit” and it would come back with, etc. “ALT 
levels are....” etc. 


If you get that information too early, it can really lead the way you form the circles, so could it 
just be available after the first PBL session? 


Yes, it could be posted or it could be timed. 


{sixth five minutes} 


It could be set up in a way that in locks you out - i.e. you have to do a patient history before you 
do a differential, before you do lab test results. The computer looks at it and says “there’s no 
differential in here so we’re not giving you any lab tests”. But there’s also a human factor in all 
this. You have to go and be prepared to talk about some of this stuff with your instructor, and if 
they say “you have no patient history there and your lab tests are filled out” - you guys didn’t do 
the case. 


For group participation - there are always going to be a few people that aren’t going to do 
anything on the web - is there a way to make it mandatory to submit something? 
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That could be part of the participation: you don’t get marks for PBL unless you participate, 
along with attendance, you have to log in. 


The way current PBLs are - definitely people skip by, 


I’ve heard that. People getting a participation mark when they haven’t done anything. We'd be 
able to track who is doing what. 


I wish someone would give me incentive to do work because its so easy to scrape by without 
having to do it. 


Tracking would make it mandatory for the PBL instead of just showing up. 

Are there any other features? 

This is sketch that I’ve done so far - can you think of any other features you’d like to see? 

I think a differential diagnosis one up there would be nice. Actually having a separate pod for 
that. That way, if you have it up on the screen, you could see what other groups were 
thinking. 

That’s a mechanism I haven’t quite figured out how to incorporate yet. Would you be 
interested in linking some of the things you found in patient history, and physical exam and 
laboratory to the differential in some way? For example you could say “there’s a relationship 
here” and somehow see that relationship. If you click on “my differential is Hep C” ifI click on 


that, maybe everything on Hep Cis presented visually on the screen to you. 


You could put a differential diagnosis link - saying “if there’s a temperature then you can have 
a differential infection, and then it would be linked because it would have the DDx right there. 


Or if you just have another circle there and then have it surrounded and not have it linked. 
Are you making it too easy for us if you make the link - because part of the skills we need to 
have is to look at the global situation and then say “differential” So if there’s a link then its just 
too easy. 


You would create the link itself. We would provide that. 


As soon as you put in a differential it would tell you what lab test to order. I think you need to 
think about what lab test to order. 


One thing for the labs though, would be if there was a link to normal ranges, or lab tests and 
what the indications are the for the lab tests. I find that’s very difficult. 


Are there any more quick comments before we go% 


I personally feel much more positive about it after I saw the sketch. 
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I think doing the paper thing is really time consuming, but I think our skills on computers are 
much faster. We can manipulate all this information must faster. I hope that you see something 
valuable in this. 


I think it will be really useful. 


-END- 
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{first five minutes} 


Some groups had family doctors doing their sessions who knew less than the people in actual 
group. 


I was amazed to listen to you guys talk about specialists knowing less about the case than you 
guys do. 


We just did this material 5 months ago. 

Its not remembering, its understanding. 

We know exactly what to look for. 

You knew the kids were important and the daycare was important, but we didn’t know why. 


Kids in daycare - massive scene for infection - so if somebody has an infection - probably the 
first place to start looking for a source. 


What do you think about PBL right now. 


PBL is a lot different than the group learning we’re doing in Neuro or Endocrine. The only 
PBL I found valuable in the last two years was Cardiac. The rest were just useless. I think it 
depends on your preceptor as the PBL. The ones where you go in and they give you a case 
and they work through it and they explain how it works and you’re done that day - I find much 
more valuable than “go off and research this” and talk about all these different, almost 
unrelated topics at times when you're doing this - its all over the place. 


Something like this may help structure that and the information is kind of down on the board 
and if you do miss a major key concept then they may bring that. It would prevent them from 
going “oh yes - we're doing GI, but there’s a really interesting respiratory condition that can 
cause GI problems” Do really want to go look that up right now? 


That’s what you find - you end up looking up a lot of stuff that was peripheral to what was 
going on. 


What was driving those tangents? 


People in the group and sometimes the preceptor. When someone in groups brings up 
something irrelevant, I’ve been in groups where the preceptor will say “well its really not 
relevant to this case - its an interesting topic - you might want to read on it”, where the next 
preceptor will say “you should go look that up” even though its not relevant to the case. It 
would help focus a group if something like this was done in terms of that, but I think also, on 
the “con” side it is majorly time consuming - I don’t know if using these tools would be part of 
the daily activity or not, but I find writing on the board goes a lot quicker. 


You can write on there and prioritize stuff really fast, just by writing 12.3.4, epesiGe It 
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If this was on line, in my mind, yowd go in, you'd have a bunch of buttons on the side and 
you'd have the case written out in front of you and if you want to bring up notes for patient 
history, then you could make sub-notes from that. That would be really useful for me. 


I still prefer to write it on the board - because I think if you're doing this physically - it takes a 
lot of time to write these things out whereas we’d already figured out everything we wanted to 
know just by talking about it, and we could put it up on the board in point form and draw 
things and circle things. 


I think the problem that we’re running into as well - because we kind of covered this 
information before - so we all already have a framework we work by when approaching a 
problem - if you picked someone in first year and made them do this GI stuff they'd have no 
idea what they were looking for, or what could be causing it. They’d be forced to try to build a 
framework. So something like that would be useful when you're just learning something. It 
would force you to build a framework by which you could hang stuff on. 


I think something like this would be more useful once you've learned all the material and this 
would be kind of synthesizing all that material. You've learned about all these various GI 
conditions, and this would help you figure out - use the same approach to everything. 


Writing on the board or doing this is the same thing. When we do it - you write on the board - 
its faster - I can see it - I can circle things - I’ve got this written. 


If everyone did this at home on the computer, and it was all the prep you had to do for this 
class, and then the tutor would come and you'd discuss questions you had, and they'd maybe 
add aspects to the case. We’d come up with a diagnosis and then you'd have to come up with 
a treatment plan with the tutor - then its not redundant. | think to sit as a group at a computer 
_ | think it would be better to do it alone or with just a couple of people. 


I prefer to work by myself, and I wouldn’t have a problem looking at this tool the night before, 
looking at the case, highlighting key points, and forming a diagram and printing it off to take it 
to PBL to look at or discuss what I thought was important. 


That would be a good way to use it. Then you wouldn't have the PBL going on and on and 
on. Come and talk about it, go look things up and come back and talk some more. I really 
like this - if you can give the patient history out bit by bit, just because when you're doing a 
physical exam - so what do we need to do? Now we need to think through it. Whereas here 
the history is here - you don’t really learn to figure out what I need to ask and this leads to 
another question that I need to make a framework for. 


I think that’s kind of a problem at the start too because everybody, when you read that, you 
immediately conceptualize everything that’s important and you start categorizing it, but its 
when you go to label it up here - you have to write it down. But when you read this case, you 
see what’s important - if you look down probably every single person underlined key words in 
this handout - and this is kind of how we summarize and the way we do stuff, then we write key 
stuff on the board and just go. 
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This feels really weird to me - it feels like I have to write every little detail down. I just read 
stuff and I think “Okay this is what I think it is “. I’m confused by it, almost. 


If you could click it on a computer screen just as you read it, click-click “add this to history”, 
etc. or highlight something and drag it or pop it into a different screen - into your patient 
history screen then you could organize from there. 


Once you’ve read through the history after highlighting stuff and dragging and popping then it 
will organize it better and you can organize the way you want. Like on PowerPoint where you 
can assign levels to it. 


If you could do something like that it would be good. 
That’s probably why we feel weird about this because we re writing everything out. 
{second five minutes} 


I feel weird about the spread-outness design. I like lists. When things are all in a circle to me 
I find it very confusing. I like how its “nodal” so you have different subdivisions, but it looks 
scattered to me. 


To build a list you already have to have it prioritized. 
Yes but you can change things around is fine to me but its the whole “web” thing. 


I agree with you in a lot of ways - it does seem awkward - but its the first time you're using a 
tool so we kind of try to get used to using the tool and figure out how we're going to prioritize 
stuff , but I think once you get used to using the framework you'd be okay, this is a big bullet, 
this is a small bullet, etc. or if you’re reading the thing, you have your three categories, 
everything you highlight goes into a place, then you get to click and drag stuff around and you 
can reorganize it your way. 


The patient history doesn’t have to be a circle. Its a heading - so under here I have “family 
history” and a list of stuff and here I have “past medical history” and my list of importance. 
You can sort of organize it I think if you leave it a little open ended about how they like to do 
it, Give them a couple of formats. 


Id like to leave it as much open ended as possible. I'll show you what I’ve got here as one 
suggestion - there could be others, but... This is very similar to what you have experienced and 
there are basically three tabs that control everything, and I haven’t thought about all the 
features, but you might click on a tab here and get a diagram and say “in this palette I can create 
notes and do some of the ranking things. I might create a node that’s called “untitled” but we'll 
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call “chief complaint” its a high priority, and you type in the title of that node, and why is that 
node important - and if so, type the rationale, then link to internet resources if you want to. 
Then you can build by clicking and dragging onto the chief complaint, for example, nausea and 
bellyache. Eventually you could map out the whole thing this way and there is also this palette 
that will allow you to zoom in on certain pieces of information if you need to. If you’ve mapped 
this out and you’ve done this really quick, and maybe you go back and fill in some information 
on your own - now! want to look at what we have tagged as very important and then look at the 
stuff that I am responsible for, then click on these nodes and fill out the information that Iam 
responsible for so I can sort it that way. That’s one possible feature that the computer allows 
you to do. The other thing we thought about including is more of a text document that 
corresponds to the diagram. We have patient history as a major node and then a sub-node of 
social history, then little nodes. Then you can say “the facts about this are X and the rationale 
for why that is important”....and you could edit in this mode if you felt more comfortable here 
and you could add notes to the diagram or change what you put in the diagram - you can flip 
back and forth in between the nodes. 


And that would actually, at least at the end, you know find that in PBL everybody talks about a 
bunch of stuff, and unless you are constantly taking notes you walk away and its like “great - I 
learned this bit of information, but I certainly don’t remember or know what you really 
investigated unless I took copious amounts of notes. So here if all that information gets 
inputted into this, you’ve got it all in the end. 


Print this out, maybe the instructor would want it submitted - he wants to see how you've 
grouped it - you can print out the diagram if you want. It will all be in a log-in format too so 
there will be an accountability that I understand doesn’t really exist in PBL right now. 


{third five minutes} 


Here, you could say “I’m sitting down in a group and I’m sitting here with these three people and 
we’re working on this”....you could track the hours that you’re spending. 


Another feature that we thought about is to compare across groups (i.e. this group did this, this 
group didn’t do anything, this group mapped out major notes and ignored physical history and 
went right to the physical exam and test results which is a big no-no. So you can compare what 
other groups are doing, and I understand that’s a bit of an issue. 


If you’re in Group 3 you're missing out a lot. If you could access what Group One did in 
terms of test results and look at why they did certain things... 


And what’s interesting is you always end of with certain blocks or you know there are one or 
two certain good preceptors from certain groups and you're stuck in a bad group but you 
don’t know what they discussed that day. So doing this, you could go look at theirs and say 
“we never discussed this - it is important”. If they post a case on the web, the biggest problem 
when you're reading stuff is - when you're reading a Word Document and then you have to 
pull stuff out of it, so you’re constantly switching screens back and forth between your Word 
Document. It would be nice if you guys could integrate something where you could highlight 
it and you could have a macro where you could just highlight and it would cut and paste and 
immediately I could just go “they have stomach pain” - highlight it and click the button. And 
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it would take all my symptoms or important facts that I pull out and drop them into a drop 
box. and then when I’m prioritizing, just take the drop box and go “boom” - look down, go 
“oh this is social history” or whatever and prioritize it based upon that. 


If you’ve got to open a palette and put it in it takes forever, and that’s just really time 
consuming, whereas I can sit down and underline the stuff - that’s all I need. 


I could see having three drop boxes for patient history, physical exam and labs or something, 
and then you just look at the history and drag the test. 


Or have a split screen - you have your text, your source, highlight the test and drag it. 


The biggest thing is making it fast. If we were reading this sheet, for instance, we could read 
“white eyes” - well its going to be a symptom of jaundice but we know its going to be a 
symptom of whatever type so we could just click the symptom button which would take you to 
the symptom drop box, and then something else, go down to “33 year old” that’s more 
dealing with social history - have a social history button which takes you to the social history 
drop box really fast. This way I don’t have to waste any time typing stuff out, because the 
more time I have to spend entering it, then the less time I want to look at it, because I can 
write it down faster than I can interact with the interface. And if that’s the case - you have to 
find something that’s equivalent to using the time on paper with the interface. 


I like this. 


That’s sort of asketch. Its not the end result of the interface. What I’m doing here is collecting 
data here to try to figure out how to do it. 


So is this something that its set up so we can do it beforehand do you think, because then.... 
and discuss it in PBL also. 


But then the part where there were different groups up there, its not going to help one group 
having a better preceptor because nobody’s going to fill the information out afterwards. 


Or if we were in small groups, just like this, had a tutor, worked on it just like we did today 
with one person punching it into the computer, and then assign it and then say “okay next time 
we get together with our tutor...” 


{fourth five minutes} 


All you have to do is say this has to be handed in and completed by the end of the small group 
session and submitted via computer, then you end up with everything put in. 


But there is no reason why this faculty couldn’t do something like this even with projectors in 
here, and have lap tops. Somebody in your PBL session signs it out, walks it down to the 
room, you hook it up - takes 2 seconds to do? 


But you see what youre doing to Joel - we re reading six symptoms, he’s hearing them all in a 
row and he’s trying to fill out cards and that’s what its like. You get 8 people in a room and 
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everybody’s yelling out symptoms and regardless whether you're writing it out on a board or in 
a computer, it has to be fast because somebody’s doing it. Like I said if you could just 
highlight and click....that’s faster than me doing it myself. 

{fifth five minutes} 


But a bigger group than this would make things even more difficult. 


That’s another issue I wanted to talk to you guys about. Right now youre in groups of 12 or 14, 
right? 


Depends on how many preceptors we have. 

And how do you feel about the size of that group as it is now? 

I think its too big. 

I think its good in some ways because those people who don’t prepare, you at least make sure 
that you get a few people in your group that are well prepared and know this stuff, but on the 
other hand, you get too many people that just sit back, then that don’t prepare, because they 
know someone is coming to the table with something. 

What would be the optimal size of group? 

I would say 5 or 6 people. 

I take it that the whole plan is to increase the whole PBL content and why? 

Hopefully, that was one of the major initiatives in the new curriculum. 

Was it just basically just following the “Jones” - somebody else did it so we're going to do it. 
When they get accreditation, one of the things every year, is people want more.... 

But why - what’s the point? 

It’s what people want. They figure it enhances learning. 

But they've done studies that shows that it doesn’t make a difference. They’ve taken people 
and randomized them to a PBL stream to a straight lecture format, and they come out with the 
same knowledge and they are exactly the same at the end. So why are they doing this? 

I think there are flaws in the studies - like you’re educated in this stream, and you’re educated in 
this stream, and then you're examined in the same way and the question is maybe people who 
are doing the PBL, and this is the theory behind it, is that they are better thinkers and better 
problem solvers about process and those kinds of things - the major things - that people really 


think is more important than being a doctor - not assuming that your recall and your more basic 
science stuff and being able to think and gather evidence on your own is the most higher order. 
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If that is the goal of this then groups and tutors aren’t important because that’s something that 
in small groups that we can do together that you’d have access to somebody to go ask 
questions to at some point in time, and you can do on your own time as a project to be 
submitted is probably more valuable than sitting around with 12 people. Because lots of times 
tutors come up and say “blah blah” and you don’t really do a lot of problem - solving, you 
collect information and then go “well, we don’t know” or go look it up - you don’t have a 
process. 


That’s the whole goal of PBL is to encourage clinical thinking, problem thinking and decision 
making and if a preceptor is up there lecturing to you it defeats the whole purpose. 


I don’t think I’ve actually encountered a PBL where any of that has really occurred, so it seems 
kind of like a..... 


Did you feel like that happened here today a little bit? 


Well, you do - this does work better - you go from Step A to Step B to Step C - logically I’ve 
got this information and its a systematic approach. But here you wouldn't need groups - you 
could have a group of 4 or 5 students that could work on this problem - you wouldn’t need 
any more and you could sit and do this almost at any time, as long as there's Dr. EA etoeOusl 
don’t understand this concept could you please take 10 minutes - could we email you to 
explain this”. 


In Pulmonary we had lots of groups but only a few tutors that circulated. I think if we had 
something like that again, you wouldn’t need a specific tutor assigned to each group. 


Yes, because you do it on your own. In Pulmonary, 95% of the time we sat and did nothing 
anyway without the tutor there. 


[sixth five minutes} 


And in some PBLs are not allowed to bring text books or whatever. I think if we have the 
resources here its going to be like real life. In real medicine you have the resources. | think 
PBL should be how you deal with things on the wards. 


You’ve got 4 students, an internet connection, you can look stuff up, you’ve got to hand it in 
by this day, here’s where you can get information, here’s a person you can talk to explain 
whatever you're missing. 


I'd really like to build in some sort of communication function so you could email the preceptor 
that’s responsible to you. And that would be one of their goals. You’re all in their class, but you 
have to take some time during the day to answer emails from students. There needs to be a 
balance there - because I know they get swamped. 


The things is - you could have groups of 4 - you could have a tutor for 8 groups and you could 
have an assigned time to work on this problem where the tutor is at a computer wherever and 
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we just send him emails when we’re confused about something - he can send it back during his 
office hours. 


And maybe you’d have to develop your questions as a group to submit so you’re not sending the 
same questions. 


Right now we have PBL slots built into the schedule. If we were allotted specific time to PBL 
rather than people trying to schedule them outside of class. 


This would probably make PBL a lot more valuable as a learning tool. 


I like having this - honestly there was a time in PBL when there were no questions - just a case 
presentation - and you had to decide what you wanted to do and what you wanted to look at. 
I like this because it gives you something to work with. 


And this format can be played with. Somebody suggested “well what if we looked at the 
relationship to pathology, anatomy and physiology and structure it that way. 


You could probably do that and you could probably have different cases doing it different 
ways. One case you’re really concerned about getting your history out and your physical 
exam, and making your differential and figuring out what it is, and the next case you want to 
know what the problem is now, these are the symptoms, what was the cause, what was the 
anatomy that was effected etc. “here’s the diagnosis - work backwards” type of thing. 


I like the same approach to every single problem. On the side if there was an assignment that 
asked questions like “where did yellow eyes come from” , why is his skin yellow’ - other 
questions. 

Sounds like you guys are really in favor of this? 

If you're going to replace PBL with this, then yes totally. 

I don’t know if its a straight replacement..... 

This would make PBL better though. 

That’s my goal is to try. 


Right now every block has its own system. If you had this for every block for some of the PBL it 
would just kind of unify it all. 


Another good thing to point out about this is that it is quite structured and organized for those 
people that are weak in problem solving or thought processes, e.g. somebody may have come 
up with this diagnosis but it could have been really scattered. In terms of this it organizes 
things really well for them. Comments could be made. 

{last five minutes} 
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It was interesting in one of the other groups where they were working on a patient history and 
somebody else in the group was saying “we've got to do this in physical exam and that...” and 
jumping way ahead, and the group said “no we’ve got to focus on patient history heres 


What did you think of some of the visualizations? Did you think that having some of that stuff 
up there was helpful? 
{all in agreement} 
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Observers’ Notes 


APPENDIX D 


February 10, 2005 — Students Constanza Pacher 


Presentation: 15’ 


—Construct a diagram > What do you mean? 

—Researcher explained the use of nodes 

—One student don’t understand the responsibilities stickers 

—Researcher reads aloud the patient presentation and students underline keywords. 

—S1 [asking to his peers]: What are the symptoms? Should we write it on papers? |he explores the 
big nodes] 


—After a few minutes, they moved to work all together 

—Fact? What does that mean? 

—What is the heading? 

—Scott: put title in big node, specifics in small 

—A bit ofa hassle with titles/subtitles 

—They are all sitting showing their back to the whiteboard. They work on the table for about 5 or 
7 minutes before going to the whiteboard. 

—Scott: Rationale... bellyache? Nausea? 

—Scott is the one that actively works with the materials. He picks nodes and writes stuff, while the 
group gives him data. 

—No apparent doubts about the use of nodes, sizes and colours 

—S] got another big node and put a title 

—S]1: Social history? She picks up a node and write © 
—10’ after starting, Marc asked them to put stuff on the whiteboard 
—Patient history goes to the top, left corner. The small nodes are arranged as bullets. No other 
apparent relationship between the nodes. O 
—Scott and $1 conduct the exercise, while $2 and S3 work on the board. O 
—They realized there is not too much space so they moved the principal node. 

—Students talk a lot while working. @ 
—They do it all together. One asks questions, the others agree/suggest. 

—They spent 15’ or more with Patient History. They wonder, they think many po 

—Then they moved to Physical Exam: How do we break it down? System? What is the Most 
important? GI? I, IPPD? [I’m not sure about these acronyms] @ 
—They immediately asked for abdominal [not sure what this means... they asked for a test?] O 
—They look/seem happy, excited, comfortable with procedure @ 
—S2 picks a small node instead of a big one, but immediately realizes of the mistake and changed CE 


it. © 
—They asked for a glossary/dictionary: Couvoisier’s sign 
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—1:15. They haven’t used the stickers. They go on working as a group. One speaks, the rest 
listens, agrees. [it seems that the stickers would be needed online, when they don’t see each 
other... not needed here]. 

—They asked for skin exam. They put the info they get on the nodes 

—Another word to be looked up in glossary: icteric 

—Scott [asking to his peers]: Do we use the big node for something that is important? Did we do one 
for skin? 

—They decide the 3 big nodes they need: US, Blood, Stool 

— Under blood we need..., and they make a list on the big node. S3 asked: Shouldn’t we write those 
in the small ones? 

—The big nodes that don’t have subsidiary nodes, the info goes inside; while the big nodes that do 
have subsidiary nodes, the info goes in small nodes in a bullet arrangement. 

—They plan the Lab Tests very well with the help of the nodes; they do like a ‘shopping list’. 
—Researcher gives all the lab tests together. 

—1:25. They are evaluating the tests results 

—[I don’t have many notes about the discussions...] 

—They recalled that the patient is a lab technician. 

—They put the first stickers: two red in blood. [The stickers help to organize the info] 

—1:30 they are “out of clues” 

—S1 [asking to her peers] What do you want to do now? Differential diagnosis? 

—They put the “vindicate” diagram on the whiteboard. 

—S1 ‘reads’ the board [it is the first time I see one student just observing the board, as a way of 
assessing what they did] 

—For Differential diagnosis they use stickers. 

—I see that they use the red ones as more important than the yellow ones (colour coding is 
stronger than the exclamation mark). 

—S3 says: We should have done the differential diagnosis before the lab tests!! 

—1:35 Wrap up. Diagnosis on board. 


Focus Group 

—S1: It was fun. P’'m visual [she didn’t look at the board too much, they mainly worked showing 
their backs to the whiteboard, however, she was the only one giving an overall look to the finish 
diagram]. 

—In PBL sessions we write things on the whiteboard 

—We should have done the differential diagnosis first. 
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February 10, 2005 — Students Constanza Pacher 


—Scott: The process was the same. 

—S3: Good for group work. If you work alone it could slow you down. 

—S1: complaint about the nodes, hard to know which ones to use [however I found that they had 
no problem understanding the use of nodes right away] 

—A student who was working pasting things on the whiteboard says that during PBL sessions 
people just throw questions/answers without too much thinking. But this [the mapping on the 
diagram] allowed time to think. 

—Sl1: Iam ‘scattered’ in my thinking... So this helped me to organize my thoughts. 

—Scott: I do write down, or in my head... I’m visual too. 

—When researcher asked if the case was too basic, the unanimous answer was ‘no’. 

—S3 asks: The info we got from you... How would that show on screen? 

—|They are trying to come up with a system to provide students with the additional information] 
One person would be the monitor? Real time feedback? You type in the question... I would like to have 


to ask for the information. In PBL they spoon-feed you. They filter the information, they just give you 


the things you need. 
—It was good that you gave us the ‘normals’. 
—But here we got the “Couvoisier’s sign” without even asking. 
—A question about PBL dynamics: It depends on who’s doing it. Sometimes someone dominates the 
class. Sometimes teacher lectures, 
—Logistics about diagram construction online: It could be annoying for 6 people on one computer. 
Better to go home and work in groups each people from different computers [I’m filling in here, you 
should listen to the tape]. 
—Researcher asks about the use of MSN system to complement: MSN is so cumbersome for 
discussion... You miss the discussion if you don’t type fast. You have to be concise. It’s too tough. 
—Researcher asks: Once the logistics are solved, would you do it?: 

If it doesn’t duplicate the work, yes! 

If the groups are small 
—Research asks: Would you use it as a presentation tool, to show what you did? 

Everybody would have 30 points... too much... redundant. 

To compare diagrams?! [skeptic] 
—Researcher mentions the report function. 

—Scott: Personally, the first time I would use the diagram, but then I would prefer to use the 
text format [he’s the one that said he’s ‘visual’... ] 
—They asked for a ‘bullet’ for differential diagnosis. 
—They mentioned something related to link the differential diagnosis to the lab tests [I missed 
this] 
—Talking about the stickers... 

—For longer tasks or the ones that you need to do research. 
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—S]1 doesn’t like the idea of stickers. 

—Scott: I don’t know if you need to rank... 

—S3 responds: Because we always get the material easy, but if we got a lot of strange info, we 
would need to rank the info. 


—S3: This could make the PBL meeting really short. You do the work at home and then you get 
together shortly. 

—This could help to use time more efficiently. 

—Scott: 3%, 4" year... we may use the diagram as a database... [I missed the context of this 
comment] 


February 14, 2005 — Instructors Terry Palmer 


- Priority question — How does it work? Users were looking for clarification 

- Rational — Users needed some clarification on what to put for it 

- Ranking — Users needed clarification 

- Users worked well as a team 

- Priority and Node size — is there a relationship 

- Users weren’t concerned with node size 

- Each person in the group thought of things the other did not 

- Used rationale to determine the priority of a node 

- Feedback — using the nodes you can track the order in which users added symptoms 

- The users brainstormed all of the symptoms, then linked them after 

- Needed more info on abdominal pain 

- The users prioritized based on important information, they also prioritized points that 
were not relevant (nodes that eliminated a potential diagnosis) 

- Users completed one section at a time. (Patient History — Physical Exam — Lab and 
Diagnosis) 

- They recorded potential diagnosis as the worked through the problem 

- They added a node to ask questions 

- They suggested built in definitions etc.. 

- Suggested to adda notes board 

- Wanted more information on Physical Exam, and Laboratory Results 

- They felt it would be important for students to rationalize the Parent-Child relationship. 


February 15, 2005 — Students Andrea Ruskin 


students started with patient history, but did have one posting up for physical exam 

one student read out case to group 

each now reading stats more closely and making guesses as to illness 

are using colour coding to match areas posted on whiteboard 

working by consensus deciding how to list categories (have changed categories once for history) 
As deciding listing deciding how to prioritize (what is parent, what is child) 


Trying to decide what is most important before listing, not listing everything and then adjusting 
priority 


Lots of discussion before they do anything, trying to make conclusions and create an ordered 
diagram 


All participants have been writing on nodes, two closest to board are posting. participant b, c and 
e are writing the most nodes 


Participant d has very strong opinions, seems to want to lead the group 
participant a is the most quiet 
Participant b seems to know the least, has mentioned missing some classes 


participant c is writing out patient history on pad, using notepads to create sub-categories for 
history 


All are writing on pads, or on case description, but are still making decisions by consensus 


Writing what they want to ask about on notepad (allergies etc.) 
(about 20 mins in to testing?) 


Still deciding what small and large node categories should be 
concerned about keeping categories around history separate 
using proximity to keep things organized and grouped 


Fairly equal participation, all are participating in process, even though seems like one or two 
would like to be leading 


using text book to look things up when they aren't sure 


deciding if something is important before the put it under history (i.e. they decided to omit kids 
thinking it was not important, but in fact it was critical) 
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February 15, 2005 — Students Andrea Ruskin 


Are writing on nodes (under patient history) in title, but not many with much else written on 
them 


Are grouping some categories - i.e. meds and allergies were grouped together 


Participant b seems concerned about looking at particular aspects of patient history - keeps 
mentioning should record sexual history 


once feel history is complete they began to discuss next steps 

E - using notepad to write what other things they would like to find out 
returning to node on physical exam - changing large node to "Vitals" 
Now going back to use stickers to assign priority 

(running out of time at this point) 

d - making notes of possibilities, very decisive 

most of group discussing possibilities of illness while b posts lab tests 

d - posting for physical exam 


Not all are listening to each other, they are rushing now to finish - not listening to the reading of 
the lab results 


e - is writing lab results on board with marker - first time used 
now paying more attention 

using arrows to show high/low levels 

using plus and minus sign to show pos. and neg. 

one student is looking through their own notes from a binder 
have just considered differential diagnosis sheet - don't fill in 
Now are prioritizing history, exam etc. 


Finish by guessing Hep A. 


February 17, 2005 — Instructors Sandra Gabriele 


What to look for: 

* procedure 

* group dynamics 

* process 

- how they use tools 


Process 
* not used to doing this as explicitly 


Tools 
* they question the urgency stickers — single / double arrows 
- why are there different tags? “can we write on the board, without the tags?” 


Procedure 
* they decide they will both write on the board 


Group Dynamics 

- discussion back and forth, confirming symptoms, ask each other what other types of exams 
might be relevant, depending on each others knowledge 

- delegated and assigned ‘other people’ to some of the possibilities when there were questions 


General Observations 

- asked for more exams than were available in the test materials 

- did not hypothesize early in the process, but too care to look at all of the patient history, exam, 
diagnostics, thoroughly 

- after coming up with the diagnosis, went back to find out how the patient got ill (marc: I guess 
this is important for confirmation of the illness??) 

- they needed more information to do this, suspected that patient was not revealing his sexual 
history 


Participants Comments 


Mapping information visually helps to 

- identify /question the details 

- make a problem list 

- helps to organize time 

- allows them to see it and they'll be less likely to miss relevant information 


- blackboard is a good idea because it’s easy but a computer will allow recording and printing 
Diagnosis is missing from the categories 


+ should list at least three things 
- students need a roadmap 
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* should be constructing a diagnostic map 
* approach that will allow them to generalize to other patients 


* students get bogged down in details 
(look at Harrison’s textbook of Internal Medicine) 


Other Observations: 
* only used the first level of tags and wrote the rest of the information directly on the boards 


- used the stickers for delegating 
* urgency stickers? 


Both participants agree that Thursday mornings is a good time to meet for a focus group. 
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February 17, 2005 — Students Ximena Rossellé 


Testing process 
Subject A read the case aloud, while everybody highlighted the data. 


Subject C transcribed the basic patient history information to the notes provided. He uses the 
smaller dots as subtopics (i.e. JAUNDICE -> big dot, and describing symptoms like yellow eyes 
and skin as bullets). 


The subjects wrote everything down and created a diagram, organizing the information on the 
table. 


The subjects worked together, sharing the information and making collective decisions about the 
importance of the info and facts. 


After looking to the relevance stickers provided, one participant wondered about the relevance of 
the patient’s grandfather condition. Another participant consulted a book to look for more 
information about Jaundice, and see if the grandfather’s condition was relevant. 


One participant used the star symbol (he wrote it with the marker in the note’s dot) to distinguish 
important facts within the patient history. 


They placed the diagram on the board, using categories to define possible problems and 
diagnoses. They created the differential diagnose. 


After reviewing the patient history information, the participants started the physical exam. They 
ordered an abdominal exam, and “translated” the information of the test results to the notes 
provided. They posted the results on the board. They repeated the same process for the rest of the 
tests (chest, rectal, eye, skin and neural). 


After all the exams were performed, subject A proposed to look at the differential diagnose again. 
He decided to keep hepatitis on top of the list (he also proposed to use a relevance sticker but they 
finally didn’t use it). 


They haven’t reviewed all the information posted on the board again, but decided to ask for lab 
tests. 


They asked for blood test, and again made the “translation” to the notes and posted them on the 
board. 


Subject A proposed to discuss the lab analysis before continuing with the tests. The participants 
are still trying to define is it is a pre or post-hepatic problem through the lab results. 


They asked for the seriology exam 


One participant linked the notes posted on board using a marker, because both were related to 
hepatic conditions. They also kept using the star symbol to assign importance. 


Subject B marked the posted note about the children in daycare, to highlight importance. 
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All the participants started to review all the facts posted on the board (from patient history to lab 
results). They started to assign priority to them using the priority stickers. 


The participants came to a final diagnose. 


The participants wondered what would have happened if they had used the priority stickers before 
and the responsibility stickers (realizing that it could have been helpful for them). 


Discussion and comments 


About PBL: 

+ “something like this may help to structure it” 

- right now, preceptors affects the group performance 

- Some participants felt that now there is no much difference between 
PBL and a regular lecture. 

- they thought that writing on the board made the process quicker 
and allowed to visualize priority. 

+ Subject A, about having the info available on the web, clicking and displaying: 

“this could be really useful for me” 

+ “This may help when you are learning about something because 
it forces you to create a framework” 

* Subject A proposed to use this tool at home and then come to class with 
the info already processed (he prefers to work alone) 

* Subject C said that it could work for an individual preparation 

* Subject D felt like she needed to write everything down and create lists. 

“Tt feels weird... I like lists, the circles are confusing”. “Looks scattered to me”. 

- Subject C responded to subject D apprehensions saying that the way of displayn the 
information could be leaved open ended. 

- Subjects B and C agreed that being allowed to click move and highlight the information could 
help to organize the info. 

* Subject A thought that it could be helpful as a web application, because you can assign levels 
and group things. 

About having the information ready for printing, subject C commented that with this tool, 
you get all the information of PBL available (he said that normally, they have to take notes or 
they will forget all the information). 

Subject B commented that it would be nice to be able to highlight the info, copy it and paste it 
in a dropbox (instead of opening Word to do so). He also proposed to drag the text to the 
dropboxes. 

- Subject B: it would be interesting to see other groups results (and see if there’s something that 
your group didn’t discuss). 

- In general, they felt that this would be an improvement for PBL. Some of the comments were: 
“It’s a good idea”; “I like this”; “this makes PBL better”; “It’s really well structured and 
organized. It allows to see the info and comments can be made”; “I like the fact that you have 
the fact (heading space) and space to write why it is important (rationale space)”. 

They proposed to use the tool and project the diagrams in the PBL sessions (it would be faster 
and clearer than drawing the diagrams on the board. They complained about the professors’ 
ability to draw diagrams). 
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* The participants commented that right now they have too many people on each PBL group, 
and that 5-6 would be ideal. 

- About using the tools on the decision making process, the participants commented “this does 
work better”. 
They proposed, as a system, to have groups of 4 people and a tutor. They can use this system 
to analyze the case and send the results by e-mail. 

* They thought that this tool will allow them to keep the information available for them, going 
back if necessary, so they won’t forget anything. 

- They proposed a tool to ask questions (in case that they don’t have a diagnose and don’t know 
what else to do, have the possibility to ask a leading question). 
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Figure 1: Visual representation of complex information 
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Diagrams can visually represent information in a way that makes the tasks of problem solving and 
decision more efficient. The map on the left shows data numerically and it is difficult to interpret 
due to the complexity. The one on the right uses proportionate sizes of dots to express quantity. 


FIGURE 2: Action research and design process 


Problem Analysis Synthesis Execution Production Evaluation 
(isin mmm «GE 

Stage 1 Stage 2 Stage 3 

Research Creative Communication 


A general design process has been established through years of practice and refinement. Although 
the process appears linear, it is instead an iterative cycle where the results of analysis, reflection 


and evaluation stages influence the previous stages 


Figure 3: Effects of discussion on group understanding 
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Small group interactions help individuals broaden their undertanding of a complex problem. 

The large outer circles represent all that could be known and understood about a problem. Each 
member of a group comes with their own experience and knowledge base. The overlapping inner 
circles represent each unique individual. Some members already share common knowledge 
base with other members in the group and, to a lesser degree, with everyone in the group. When 
members gather, share and interpret information in order to solve a problem they reach a 
mutual understanding. 


FicureE 4: Bloom’s taxonomy of educational objectives 


Application 


Source: Wakefield, 1996, p. 409 


The Taxonomy of Educational Objectives was created by the American National Board of Examin- 
ers in 1956 in an effort to classify educational behaviours for college and university examiners. 

It shows a continuum of progression from general skills to more specific behaviours. Each stage 
builds upon the foundations created by the previous one. Early educational training focuses on the 
first three, while higher education attempts to train individuals with skills in the following three. 

In the 50 years that have followed, the taxonomy has become a basic reference for educators. 


FIGURE 5: GroupSystems’ GS |/ team decision making and collaborative thinking tool 
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Screenshots from GroupSystems’ website indicated that GS Il is a highly structured and heavily 
text based decision making and collaborative thinking environment. This type of approach encour- 
ages a very linear and sequential way of thinking that may not always be good for a complex prob- 
lem. Using the software, teams can: (1) set an agenda for work; (2) brainstorm new ideas; 

(3) organize the brainstorming concepts; (4) describe those concepts in more detail; (5) 

create a tree structure with a logical flow; (6) weigh each option; (7) tabulate the results; and 

(8) generate a report. 


FIGURE 6: Levels of reading: Overall, intermediate and elementary 


Source: Bertin, 1983, p.141 


The first diagram represents the elementary level of reading. On one particular day, the stock price 
was at one particular level. The second diagram shows the intermediate level, where the analysis 
can be expanded. Over the first three days, the stock fell. At the overall level, one can expand the 
search even wider to answer the much larger question, “What was the overall trend of the stock?” 
The stock rose. 
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FiGure 7: Groups of imposition and types of imposition 
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Maps ae Source: Bertin, 1983, p. 52 
The utilization of two planar dimensions is called “imposi- 


Q) | 
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tions”. Groups of imposition can be classified according to 
the way in which components are represented. These groups 
consist of diagrams, networks, maps and symbols. The table 
figure also shows that the components can be inscribed in 

a variety of ways for diagrams and networks. These various 
ways of inscription are called types of imposition. 


FiGuRE 8: /mplantations and visual variables 


Implantations Visual variables 


Point 


Line 


eed 
’_eeod 
a 


Texture 


Orientation 


by et a 


Hue 


peony 


Shape 


Source: Bertin, 1981, p. 187 


Implantations are the three elementary figures of plane geometry that are used to assigna 
mark on the plane. Visual variables are the graphic notations that can be applied to each of the 
implantations to distinguish relationships. 


FiGuRE 9: Gestalt principles: Closure, proximity and similarity 
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Source: Bowers, 1998 


Gestalt psychology is the way humans perceive and organize visual components into meaningful 
wholes when the meaning of each individual part differs. The three basic gestalt principles are: (A) 


closure, (B) proximity, and (C) similarity. 


FIGURE 10: Mind maps 


http://www.mind-map.com/EN/mindmaps/gallery.html 


Mind mapping is a technique developed to map issues around a central theme while taking advan- 
tage of the “full range of cortical skills” —words, numbers, images, and colour using spatial ar- 
rangement. The laws of mind mapping encourage people to develop a personal style that is mean- 
ingful to themselves. Unfortunately, as expressed in the large map above, these freeform 
expressions can be difficult for others to understand. 


FIGURE 11: Concept maps 
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Concept maps are a means of spatially organizing information. This mapping technique starts 
with a central concept and uses linking words to bridge concepts that are subordinately related. 


Information is structured in a hierarchical arrangement that forces the v 
and sequential way. If we are to examine the region of the map charted 
would read like this: “Concept maps represent organized knowledge ne 


iewer to read in a linear 
by numbers 1 through 6, it 
cessary for effective teach- 


ing and effective learning”. This approach violates the principles of levels of reading and reduces 


the possibilities to see inclusions and simultaneity. 


FIGURE 12: Patient information summary exploration 
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Photos depicting early explorations to map patient information in a network diagram using paper 
and markers. These explorations helped determine the appropriate format of a diagram to contain 
large amounts of information. 


FIGURE 13: Patient information summary exploration: networks 
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A less structured network diagram, similar to a mind map, was deemed appropriate to plot a large 
amount of information. History, Physical Exam and Tests were selected as three large headings 
around which information could be mapped. The relative distance between the large headers and 
the individual pieces of information helped identify large groups. Lines could link pieces of infor- 
mation and colour could be used to highlight important data. 


FiGurRE 14: Exploration of visual variables 


After the patient information was transcribed into text on the computer, a series of explorations 
were made using visual variables. Colour was applied consistently as a means of distinguishing 
major groups of information. In an early version (1), a different size of circles was used for each lev- 
el of information. Ina second version (2), shapes were used to help further distinguish groups and 
the proportionate size of a node grew depending on the number of sub-nodes. In a third attempt 
(3), shapes were used to distinguish each level of information. A consistent shape was determined 
to be the most readable, so a fourth attempt (4) varied the size of the circle to indicate the amount 
of information each node contained. 


Figure 15: Final variation of patient diagram 
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In the final variation of the diagram, efforts were made to find an economical display for the infor- 
mation that would make optimal use of the computer screen. Circles were selected because they 
provide the most space for adjacent nodes when compared to other shapes. Up to nine sub-nodes 
can encircle a single node (see inset). Lines linking pieces of information were removed to reduce 
clutter. Instead, relationships are shown by the combination of colour and position of nodes next to 


one another. 
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FiGURE 18: Interface sketch 3 
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FiGurE 20: Paper prototype, version 1 
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FiGure 21: Diagram construction toolkit 


FIGURE 22: Animated computer demonstration 
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A series of animated screens demonstrated how the diagram construction could operate using 

a computer application. The screen is blank with three larges nodes at the beginning. Nodes are 
added until the whole diagram is complete. Properties assigned to nodes containing information is 
used to search and select items in the diagram. A side-by-side comparison could help students and 
instructors assess the work. 


FIGURE 23: DiagramA 
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FiGure 24: Diagram B 
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FIGURE 25: Diagram C 
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FIGURE 26: Diagram D 
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FIGURE 27: Diagram E 
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FIGURE 28: Higher-order thinking analysis framework 
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FIGURE 29: Group interaction 


The diagram construction tookit allowed the students to work as a self-directed group to diagnose 
a patient problem. A high degree of interaction occurred between the participants of each group. 


cata 


OT 


1G 


ONE AHEE 


a 


AS 


a 
raat 


Siz 
SN: 
FRG 

a 


Heal 
eis 


ae 


Non 
ok 


SN 


euttees 
Nha 


| 
¢ 


ii 


